Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi All, I'M Running Some Training Tasks Remotely Using Clearml Agent, Using My Github Repo Code. One Of The Input Arguments To My Process Is A Config Yaml File. The Pipeline Is Setup So That This Config File Is Referenced With A Relative Path Such As

Hi all,
I'm running some training tasks remotely using ClearML Agent, using my github repo code. One of the input arguments to my process is a config yaml file. The pipeline is setup so that this config file is referenced with a relative path such as python my_training_script.py --config ./configs/my_config.yaml
If I manually edit this config file through the app (without committing it to my github repo), and run the task using enqueue functionality, is there a way to reference this file to my process? To clarify, the file only exists in the app, not in the repo.
Thanks in advance

  
  
Posted 2 years ago
Votes Newest

Answers 4


If so, what do you mean by "is there a way to reference this file to my process"?

  
  
Posted 2 years ago

Hi ResponsiveHedgehong88 , let me see if I get it straight, you have a my_config.yaml in your repo. Then you do something like task.add_configuration(my_config.yaml) and have it logged as a config object, then you modify the config object and rerun (so now it takes the configuration from the configuration object you modified in the UI, rather than the file in the repo). Am I understanding the setup correctly?

  
  
Posted 2 years ago

Hi, what we do is indeed override the configuration so instead of taking the info from the file, we take from the input from the UI. If you're using the content of the file in other places directly, you might end up with different results.
If you can have a short code snippet to show me how you're reading \ tracking the file, I can maybe better help and provide a suggestion of what can be done!

  
  
Posted 2 years ago

Hi, thanks for the answer. Loading manually set params is fine as you have explained, but I would need the file reference as well. The thing is, I'm using both the config file contents (params) and the file path (re-loading some params in other processes that don't share all the resources).
Does what you are saying mean that the file I have modified in the app overrides the existing file under the same file name in repo clone on my training instance?

  
  
Posted 2 years ago