Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Every One, What Is The Algorithm To Schedule A Task After Pulling It From A Queue In Agent? I Am Using Clearml Open Source. Thanks In Advance.

Hi, every one, what is the algorithm to schedule a task after pulling it from a queue in agent? I am using ClearML Open Source. Thanks in advance.

  
  
Posted 2 years ago
Votes Newest

Answers 9


Hi, SuccessfulKoala55 Maybe this is not a good question. In other words, docs introduce that ClearML Open Source supports orchestration, how can I found the relating codes? And what the role of clearml-agent in orchestration, a combination of kube-scheduler and kubelet?

  
  
Posted 2 years ago

Hi FloppyDeer99 ,

after pulling it from a queue in agent

What do you mean exactly?

  
  
Posted 2 years ago

Hi FloppyDeer99 ,

In other words, docs introduce that ClearML Open Source supports orchestration, how can I found the relating codes?

You can find many examples https://clear.ml/docs/latest/docs/getting_started/mlops/mlops_first_steps/ , if you have a specific use case you want to check, please share and I can send an example of it.

And what the role of clearml-agent in orchestration, a combination of kube-scheduler and kubelet?

ClearML agent is an ML-Ops tool for users to run jobs on any machine, with or without any changes (full arch https://clear.ml/docs/latest/docs/getting_started/architecture/ )

  
  
Posted 2 years ago

Sure, with clearml and clearml-agent you get autoscaling for your machines (with monitoring) and automation for your tasks that will handle all for you (docker images, manage the credentials …).
There are much more parts in the system, so maybe you can share a use case so I can help you with it?

  
  
Posted 2 years ago

Hi TimelyPenguin76 , thanks for you reply. I still confuse about ClearML’s orchestration. Could you describe it in detail?

  
  
Posted 2 years ago

For example, I have a lot of tasks in a queue, but there is a default agent(services, mode = daemon) after using helm to deploy the ClearML in Kubernetes. As far as I know, agent will pull and execute the task in specified queue one by one, which results in other tasks are blocked. However, the physical resource is enough in cluster. So how can ClearML to improve the efficiency of execution? I can solve it by implementing a custom scheduler which is used to watch the queue, pull the task and send it to a remote environment. This environment will prepare the prerequisites and run agent in execute mode. But docs introduce the ClearML Open Source has orchestration feature. I want to find out whether the orchestration support the above situation?

  
  
Posted 2 years ago

Hi, TimelyPenguin76 , could you help me?

  
  
Posted 2 years ago

Hi FloppyDeer99 ,

It depends on you setup:
if you have on prem machines, you can start more than one clearml-agent on the machine with the resources and assign for example each gpu on the machine to a https://clear.ml/docs/latest/docs/clearml_agent#docker-mode . You can have the same for cloud machine, and if you are using the AWS you can run the https://clear.ml/docs/latest/docs/guides/services/aws_autoscaler/ as a service. K8S: there is a great example for k8s glue https://github.com/allegroai/clearml-agent/blob/master/examples/k8s_glue_example.py

I can solve it by implementing a custom scheduler which is used to watch the queue, pull the task and send it to a remote environment. This environment will prepare the prerequisites and run agent in execute mode.

Each of the above should do it for you 🙂

  
  
Posted 2 years ago

Thanks TimelyPenguin76 , I think I got it, but I have new questions when reading README of ClearML Agent. And I gave my questions in community just now, could you have a look at them? https://clearml.slack.com/archives/CTK20V944/p1626251661217200

  
  
Posted 2 years ago
451 Views
9 Answers
2 years ago
one year ago
Tags