Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Am Using Pipelines (Just Starting) And I Am Checking Different Options For Overriding Parts Of Configuration Of The Base Task (Step Of My Pipeline). In The Docs For Parameter_Override One Can Find:

I am using pipelines (just starting) and I am checking different options for overriding parts of configuration of the base task (step of my pipeline). In the docs for parameter_override one can find:
Pipeline Task argument (seePipeline.add_parameter) parameter_override={'Args/input_file': '${pipeline.<pipeline_parameter>}' }Can I do a similar thing for configuration, i.e., do configuration_override with name "General/my_parameter_name" so that only this part of the configuration will be updated?

  
  
Posted one year ago
Votes Newest

Answers 7


ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)

Yep sounds good, something like this?
from clearml.utilities.dicts import ReadOnlyDict, merge_dicts overrides = {} task.connect(overrides) configuration = {#stuff here} task.connect_configuration(configuration) merge_dicts configuration.update(overrides)BTW: this will allow you to override any specific configuration (even nested) with "General/key/sub"
(do notice that casting might be tricky, assume you will get everything in String)

  
  
Posted one year ago

Hi UpsetTurkey67

"General/my_parameter_name" so that only this part of the configuration will be updated?

I'm assuming this is a Hyperparameter not a configuration object (i.e. task.connect not task.connect_configuration), if this is the case then Yes 🙂

  
  
Posted one year ago

it is a configuration object (line of my code:
config_path = task.connect_configuration(config_path)

  
  
Posted one year ago

which is probably why it does not work for me, right?

  
  
Posted one year ago

ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)

  
  
Posted one year ago

which is probably why it does not work for me, right?

Correct, you need to pass the entire configuration (it is stored as a blob, as opposed to the hyperparameters that are stored as individual values)
:param configuration_overrides: Optional, override Task configuration objects. Expected dictionary of configuration object name and configuration object content. Examples: {'General': dict(key='value')} {'General': 'configuration file content'} {'OmegaConf': YAML.dumps(full_hydra_dict)}

  
  
Posted one year ago

I did something similar to what you suggests and it worked, the key insight was that connect and connect_configuration work differently in terms of overrides, thanks!

  
  
Posted one year ago