Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Trying To Use Omegaconf With Task.Connect_Configuration And I Get The Following Error:

Hi, I am trying to use omegaconf with task.connect_configuration and I get the following error:
>>> OmegaConf.create(task.connect_configuration(config_dict)) omegaconf.errors.ValidationError: Object of unsupported type: 'ProxyDictPostWrite'How can I fix that? Is there a recommended way of using omegaconfig with task.connect_configuration?

  
  
Posted 2 years ago
Votes Newest

Answers 28


Regarding connect_configuration() , reading into the docs I see that this method needs to be called before reading the config file
https://clear.ml/docs/latest/docs/references/sdk/task#connect_configuration

  
  
Posted 2 years ago

I am not using hydra, I am reading the conf with:
config_dict = read_yaml(conf_yaml_path) config = OmegaConf.create(task.connect_configuration(config_dict))

  
  
Posted 2 years ago

Same, it also returns a ProxyDictPostWrite , which is not supported by OmegaConf.create

  
  
Posted 2 years ago

From the looks of this example this should be connected automatically actually
https://github.com/allegroai/clearml/blob/master/examples/frameworks/hydra/hydra_example.py

  
  
Posted 2 years ago

Doing it the other way around works:
` cfg = OmegaConf.create(read_yaml(conf_yaml_path))
config = task.connect(cfg)
type(config)

<class 'omegaconf.dictconfig.DictConfig'> `

  
  
Posted 2 years ago

But I am not sure it will connect the parameters properly, I will check now

  
  
Posted 2 years ago

Yea, the config is not appearing in the webUI anymore with this method 😞

  
  
Posted 2 years ago

I have a custom way of reading the config file

  
  
Posted 2 years ago

with open(path, "r") as stream: return yaml.load(stream, Loader=yaml.FullLoader)

  
  
Posted 2 years ago

This allows me to inject yaml files into other yaml files

  
  
Posted 2 years ago

it would be nice if Task.connect_configuration could support custom yaml file readers for me

  
  
Posted 2 years ago

So I need to have this merging of small configuration files to build the bigger one

  
  
Posted 2 years ago

and then call task.connect_configuration probably

  
  
Posted 2 years ago

Otherwise I can try loading the file with custom loader, save as temp file, pass the temp file to connect_configuration, it will return me another temp file with overwritten config, and then pass this new file to OmegaConf

  
  
Posted 2 years ago

I see the issue. SuccessfulKoala55 , what do you think?

  
  
Posted 2 years ago

Hey SuccessfulKoala55 , unfortunately this doesn’t work, because the dict contains others dicts, and only the first level dict becomes a dict, the inner dicts still are ProxyDictPostWrite and will make OmegaConf.create fail

  
  
Posted 2 years ago

Hey JitteryCoyote63 , the ProxyDictPostWrite object is derived from dict - can't you just do config = OmegaConf.create(dict(task.connect_configuration(config_dict))) ?

  
  
Posted 2 years ago

Ok, so what worked for me in the end was:
config = task.connect_configuration(read_yaml(conf_path)) cfg = OmegaConf.create(config._to_dict())

  
  
Posted 2 years ago

ProxyDictPostWrite._to_dict() will recursively convert to dict and OmegaConf will not complain then

  
  
Posted 2 years ago

but then why do I have to do task.connect_configuration(read_yaml(conf_path))._to_dict() ?
Why not task.connect_configuration(read_yaml(conf_path)) simply?
I mean what is the benefit of returning ProxyDictPostWrite instead of a dict?

  
  
Posted 2 years ago

That's because ClearML needs the hooks in that class to make sure any changes you make in that data structure are propagated back to the server when you're not in remote mode

  
  
Posted 2 years ago

erf, I have the same problem with ProxyDictPreWrite 😄 What is the use case of this one ?

  
  
Posted 2 years ago

This one doesn’t have

_to_dict

unfortunately

Well, I guess we can add it...

  
  
Posted 2 years ago

This one doesn’t have _to_dict unfortunately

  
  
Posted 2 years ago

I ended up dropping omegaconf altogether

  
  
Posted 2 years ago

Yes that’s what I did initially, but eventually I decided that it’s too much complexity added for nothing really, I’d rather drop omegaconf and if one day clearml supports it out of the box take advantage of it

  
  
Posted 2 years ago

👍

  
  
Posted 2 years ago
674 Views
28 Answers
2 years ago
one year ago
Tags
Similar posts