I did something similar to what you suggests and it worked, the key insight was that connect and connect_configuration work differently in terms of overrides, thanks!
ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)
Yep sounds good, something like this?from clearml.utilities.dicts import ReadOnlyDict, merge_dicts overrides = {} task.connect(overrides) configuration = {#stuff here} task.connect_configuration(configuration) merge_dicts configuration.update(overrides)
BTW: this will allow you to override any specific configuration (even nested) with "General/key/sub"
(do notice that casting might be tricky, assume you will get everything in String)
which is probably why it does not work for me, right?
Correct, you need to pass the entire configuration (it is stored as a blob, as opposed to the hyperparameters that are stored as individual values):param configuration_overrides: Optional, override Task configuration objects. Expected dictionary of configuration object name and configuration object content. Examples: {'General': dict(key='value')} {'General': 'configuration file content'} {'OmegaConf': YAML.dumps(full_hydra_dict)}
which is probably why it does not work for me, right?
it is a configuration object (line of my code:config_path = task.connect_configuration(config_path)
Hi UpsetTurkey67
"General/my_parameter_name" so that only this part of the configuration will be updated?
I'm assuming this is a Hyperparameter not a configuration object (i.e. task.connect not task.connect_configuration), if this is the case then Yes 🙂
ok, I will do a simple workaround for this (use an additional parameter that I can update using parameter_override and then check if it exists and update the configuration in python myself)