Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
After Saving Models During Training I Cant Reuse The Same Task To Continue Training Is There Any Way To Continue Task Even Though There Are Models Saved? Alternatively Is There Any Way To Make Torch.Save Not Automatically Save The Model?

After saving models during training I cant reuse the same task to continue training
Is there any way to continue task even though there are models saved?
Alternatively is there any way to make torch.save not automatically save the model?

  
  
Posted 2 years ago
Votes Newest

Answers 6


If it interests you this seems to work
last_task = Task.get_task(project_name="playground-sandbox", task_name='foo2') task = Task.init(project_name="playground-sandbox", task_name='foo2', continue_last_task=last_task.id if last_task else None)

  
  
Posted 2 years ago

StaleButterfly40 , alternatively you could use auto_connect_frameworks=False
https://clear.ml/docs/latest/docs/references/sdk/task#taskinit
So torch.save won't automatically save the model, however, you will not get the scalars/metrics automatically as well.

  
  
Posted 2 years ago

CostlyOstrich36 Thanks!
But it seems like this only works if I am running both times from the same machine because clearml is not checking if task exists in server - it is checking if it is in cache_dir

  
  
Posted 2 years ago

StaleButterfly40 , it looks like there might be a good solution for your request. In the previous link I provided, there is a parameter ' continue_last_task' that should work for you 🙂

  
  
Posted 2 years ago

I want everything to appear in the same experiment (e.g scalar metrics)

  
  
Posted 2 years ago

StaleButterfly40 Hi!
You could clone the original task and edit the input model of the new task as the output model of the previous task 🙂

  
  
Posted 2 years ago