Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I Have Some Clearml Agents On Gcp And Sometimes The Instance Seems To Reboot Making The Experiment Fail And All The Progress Is Lost. What Is The Best Way To Resume An Experiment?

Hi! I have some ClearML agents on GCP and sometimes the instance seems to reboot making the experiment fail and all the progress is lost. What is the best way to resume an experiment? 🥲

  
  
Posted 2 years ago
pic
More
GrievingTurkey78
0 × 1
Report
Votes Newest

Answers 3

Hey GrievingTurkey78 ,

Please take a look here : https://clear.ml/docs/latest/docs/references/sdk/task#taskinit

I think what you're looking for is this:
Task.init(.., continue_last_task=True )

Just search for this parameter for more more info 🙂

  
  
Posted 2 years ago
pic
More
CostlyOstrich36
0
Report

Thanks 🙌

  
  
Posted 2 years ago
pic
More
GrievingTurkey78
0 × 1
Report

Hey CostlyOstrich36 sorry to ping you! Let's say I enqueue multiple experiments on a couple of agents and one of them fails. Is it possible to restart the experiment from the UI using the latest checkpoint? What if the experiment gets assigned to the other agent? I am not sure how the continue_last_task flag would help in this case.

  
  
Posted 2 years ago
pic
More
GrievingTurkey78
0 × 1
Report
Write your answer
957 Views
3 Answers
2 years ago
one year ago
Tags
clearml