Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi All - I Am Expeiencing Some Weird Behavior Using Clearml Experiment Tracking With Hydra Configurations. My Hydra Omegaconf Configuration Object Is Not Always Being Picked Up, And I Am Unable To Consistently Reproduce It. Sometimes I Get The Omegaconf

Hi all -

I am expeiencing some weird behavior using clearml experiment tracking with hydra configurations. My hydra OmegaConf configuration object is not always being picked up, and I am unable to consistently reproduce it. Sometimes I get the OmegaConf configuration object under the "configuration" tab, and other times it is just missing. It does not seem to come and go completely on random, usually has consistent behavior for some time (hours/days) and then changes. Anyone with similar experience? any idea how I best debug this? any input is greatly appreciated 👍 I am using clearml v1.14.4, have seen the same behaviour in v1.14.1

Posted one month ago
Votes Newest

Answers 2

@<1523701205467926528:profile|AgitatedDove14> I run the experiments manually for now. It does seem I found the cause of the behaviour, though: I am instantiating an object from my own "tracker" class in my main method that holds from the clearml Task object that actually does the logging. I am doing the instantiation from my configuration via hydra.utils.instantiate method. So that means import clearml was not executed before already in my main method annotated with hydra.main :

@hydra.main(version_base=None, config_path="conf", config_name="config")
def main(cfg: DictConfig):

I now make sure the clearml package is imported before I call my main method and that seems to fix it. What confused me was that instantiating explicitly my tracker object worked, now I think that was because in doing this I would let my IDE add the import tracker statement to the top of the file and clearml would therefore be imported before reaching main method :)

Posted one month ago

Hi @<1655744373268156416:profile|StickyShrimp60>

My hydra OmegaConf configuration object is not always being picked up, and I am unable to consistently reproduce it.
... I am using clearml v1.14.4,

Hmm how can we reproduce it? what are you seeing what it does "miss" the hydra, i.e. are you seeing any Hydra section? how are you running the code (manually , agent ?)

Posted one month ago