Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Looking At Clearml As An Option To Automate Our Training Pipelines. However, From Reading The Documentation I'M Confused If Clearml Can Do What We Want. In Essence, I Would Like To Understand The Methods Of Queuing A

Hi, I'm looking at ClearML as an option to automate our training pipelines. However, from reading the documentation I'm confused if ClearML can do what we want. In essence, I would like to understand the methods of queuing a Task at a Worker .

Say I have a script, experiment_a.py that when executed trains some model for a dataset, and I have a ClearML Worker in Docker mode waiting for tasks in the queue default . How would I as a user, enqueue and modify my script experiment_a.py such that it can be executed as a Task on the Worker ?

E.g. adding the following lines to the top of my script (like here https://github.com/allegroai/clearml/blob/master/examples/frameworks/pytorch/pytorch_mnist.py ):
` from clearml import Task

task = Task.init(project_name='experiment_a', task_name='experiment_a', task_type=Task.TaskTypes.training) Will of course create a new task and start the training locally when executed; not on the Worker . I see no examples in the documentation how what I need such that I can enqueue my script as a task at my Worker ` for execution? Can anyone tell me if that's possible, and what the steps are to do so?

  
  
Posted 3 years ago
Votes Newest

Answers 3


I think you're looking for the execute_remotely function?

  
  
Posted 3 years ago

Hi GracefulDog98
As UnevenDolphin73 pointed you might be looking for https://clear.ml/docs/latest/docs/references/sdk/task#execute_remotely
Which will stop the current local process, and enqueue the task on the "default" queue, for the agent to execute.
Is this what you are looking for ?
The idea is you can run your code once in "development" mode, so you know everything is working, then from the UI (or programmatically) you can clone the experiment, edit the configuration (or anything else) and send for execution on one of the workers.

  
  
Posted 3 years ago

Thanks, that's what I was looking for

  
  
Posted 3 years ago
1K Views
3 Answers
3 years ago
one year ago
Tags
Similar posts