Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Quick Question - Does Clearml'S Task Support Subprocesses Launched Within A Script? I Have This Scenario

quick question - does clearml's task support subprocesses launched within a script? I have this scenario

  • clearml task launches a dvc script using their api
  • dvc script does the equivalent of dvc repro , which calls another python script that trains an ml model
  • the other python script launches (script2.py) another clearml task within the script, to do experiment tracking
    Ideally, I would like
  • just one task
  • parameters, scalars and git diff of script2.py reported to the clearml dashboard
    Is this possible? Moving away from DVC is not an option atm for us as there are lots of dependencies.
  
  
Posted one year ago
Votes Newest

Answers 8


@<1537605940121964544:profile|EnthusiasticShrimp49> , now that I have run the task on remote, can I copy the artefacts/files it creates back to my local fs?

Lets say the artefacts are something like
artefacts = [checkpoint.pth, dvc.lock, some_other_dynamically_generated_file]

  
  
Posted one year ago

Hey @<1535069219354316800:profile|PerplexedRaccoon19> , yes it does. Take a look at this example, and let me know if there are any more questions: None

  
  
Posted one year ago

Thanks! so it seems like the key is the Task.connect and bubble up params to original task, correct?

  
  
Posted one year ago

I do change the task and the project name, the task name change works fine but the project name change silently fails

  
  
Posted one year ago

Yes, you can do that. But it may make it harder to identify the task later on

  
  
Posted one year ago

Would I also be able to change the task name from within the subprocess?

  
  
Posted one year ago

Hmmm, my only issue there is that not all of my "artefacts" are clearml artefacts.

The files I need are models and other locally modified files that get generated by the clearml task on remote

  
  
Posted one year ago

To copy the artifacts please refer to docs here: None

  
  
Posted one year ago