Hi AgitatedDove14 ,
In OmegaConf.to_container (or yaml) there is a resolve argument "if resolve is set to True, interpolations will be resolved during conversion". Currently in clearml you save the argument after resolving and I'm looking to save them explicitly so the user will not forget to change some dependencies.
Long story short, this is done internally when you call the Task.init (I think, there is a chance it is called before)
One way of controlling it would be to have something like:Task.init(auto_connect_frameworks={'hydra': {'log_before_resolve': True}})That said, I think it will be simpler to store both (in different section of course)
Maybe "Configuration Object: OmegaConf" and "Configuration Object: OmegaConfDefinition" ?
PricklyJellyfish35 yes that's kind of what I was thinking 🙂
I still wonder if we should configure it or just have both.
Could I ask you to open a GitHub issue on this feature request, I'd love to get some input on what would make more sense to implement. Regardless it is not a major change and should be very quick to implement
Sounds good but the resolve config should be read only. Practically, in PatchHydra._register_omegaconf you can use OmegaConf.to_yaml(config, resolve=PatchHydra._log_before_resolve) without hydra key. WDYT?
(fyi: once we have a solid idea here, please open a github issue on the feature request, I'll try to see if we can push it fwd for the next RC 🙂 )
AgitatedDove14
I have the configuration after the overrides.
Hi TimelyPenguin76
Yes, but the variables were interpolated.
Which means you currently save the argument after resolving and I'm looking to save them explicitly so the user will not forget to change some dependencies.
That is correct
I'm looking to save them explicitly so the user will not forget to change some dependencies.
Hmm interesting point. What's the use case for storing the values before the resolving ?
Do we want to store both ?
The main reason for storing the post resolve values, is that you have full visibility to the actual values used in the process. Which makes it easier to compare different Task executions
wdyt?
So only the before resolving? How would you think to configure it?
Hmm you mean like overrides ?
Maybe store both before/after resolving ?
(Although that might be confusing? as the before solve should actually be readonly)
I think it will be confusing to save both
PricklyJellyfish35
Do you mean the original OmegaConf, before the overrides ? or the configuration files used to create the OmegaConf ?
AgitatedDove14 TimelyPenguin76 , any idea?
Hi PricklyJellyfish35
My apologies this thread was forgotten 😞
What's the current status with the OmegaConf, ? (I'm not sure I understand what do mean by resolve=False)
Maybe add a configurable argument that will control the resolving. What is the current process before saving the configuration? Do you directly save the omegadict or first parse it as dictionary? (and no problem I'll post)
I understand but a user who runs an experiment from the server may change one argument and not its dependency which may result in error in the good case or undesirable run in the worst case. Maybe you can add an argument to control it.
Hi PricklyJellyfish35 ,
You should have the OmegaConf file as part of the task configuration (under CONFIGURATION tab), do you have it there?
