Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Guys! How Do You Handle Tasks With A Complex Parametrization? For Example, A Script That Trains A Machine Learning Model, Where You Want To Parametrize Model Name, Hyperpars, Preprocessing Steps, Etc. So A Nested Configuration With Many Parameters Do I

Hi guys! How do you handle tasks with a complex parametrization?
For example, a script that trains a machine learning model, where you want to parametrize model name, hyperpars, preprocessing steps, etc. So a nested configuration with many parameters
Do I have to use Hydra and define a cli that supports nested parameters?
Or is there a way to pass a configuration file (yml or python code) to the task?

  
  
Posted 8 months ago
Votes Newest

Answers 6


  1. yes they will! This is exactly the idea :)
  2. yes it will store it as text file (as is raw text) notice the return value is the file you should open. This is because when running via agent the return file will contain the conf file from the UI. Make sense?
  
  
Posted 7 months ago

Lol yeah Hydra is great. Notice you still have the ability to override Hydra from the UI so you really have the best of the two worlds

  
  
Posted 7 months ago

Hi @<1691620877822595072:profile|FlutteringMouse14>

Do I have to use Hydra

You can, and then the entire configuration is fully captured by ClearML (automatically) while you can still override values with the manual "key.sub=value" both in the UI and in the CLI

Otherwise you can connect nested dict with task.connect (these will be flattened with / for sub keys).
Or you can connect configuration files ( task.connect_configuration ) and edit them as is in the UI (with override of course)
wdyt?

  
  
Posted 8 months ago

I had no idea you could do this, that's amazing thanks a lot!
Although after reviewing the options, probably managing everything with Hydra is the better way :D

  
  
Posted 7 months ago

Yeah that sounds great, look forward to try it! Thanks a lot :)

  
  
Posted 7 months ago

That's exactly what I was looking for! Thanks a lot.
Just two followup questions

  1. if I connect a dict with task.connect and I edit it in the UI, will the new values be passed to the next run?
    2)with connect_configuration, can I pass a python file containing the configuration (like, a configuration-as-code file, instead of a yml or a json)? Could I modify it from UI?
  
  
Posted 7 months ago
675 Views
6 Answers
8 months ago
7 months ago
Tags