Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Am Using Pipelines (Just Starting) And I Am Checking Different Options For Overriding Parts Of Configuration Of The Base Task (Step Of My Pipeline). In The Docs For Parameter_Override One Can Find:

I am using pipelines (just starting) and I am checking different options for overriding parts of configuration of the base task (step of my pipeline). In the docs for parameter_override one can find:
Pipeline Task argument (seePipeline.add_parameter) parameter_override={'Args/input_file': '${pipeline.<pipeline_parameter>}' }Can I do a similar thing for configuration, i.e., do configuration_override with name "General/my_parameter_name" so that only this part of the configuration will be updated?

  
  
Posted 12 months ago
Votes Newest

Answers 7


it is a configuration object (line of my code:
config_path = task.connect_configuration(config_path)

  
  
Posted 12 months ago

which is probably why it does not work for me, right?

Correct, you need to pass the entire configuration (it is stored as a blob, as opposed to the hyperparameters that are stored as individual values)
:param configuration_overrides: Optional, override Task configuration objects. Expected dictionary of configuration object name and configuration object content. Examples: {'General': dict(key='value')} {'General': 'configuration file content'} {'OmegaConf': YAML.dumps(full_hydra_dict)}

  
  
Posted 12 months ago

I did something similar to what you suggests and it worked, the key insight was that connect and connect_configuration work differently in terms of overrides, thanks!

  
  
Posted 12 months ago

Hi UpsetTurkey67

"General/my_parameter_name" so that only this part of the configuration will be updated?

I'm assuming this is a Hyperparameter not a configuration object (i.e. task.connect not task.connect_configuration), if this is the case then Yes 🙂

  
  
Posted 12 months ago

ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)

  
  
Posted 12 months ago

ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)

Yep sounds good, something like this?
from clearml.utilities.dicts import ReadOnlyDict, merge_dicts overrides = {} task.connect(overrides) configuration = {#stuff here} task.connect_configuration(configuration) merge_dicts configuration.update(overrides)BTW: this will allow you to override any specific configuration (even nested) with "General/key/sub"
(do notice that casting might be tricky, assume you will get everything in String)

  
  
Posted 12 months ago

which is probably why it does not work for me, right?

  
  
Posted 12 months ago