Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hello! A Question

Hello! A question https://clearml.slack.com/archives/CTK20V944/p1656420047506249 . Is it possible to get an untracked file uploaded when remote agent execution from --folder . is set?
We have a large set of params in a JSON file, which is inconvenient to use as params/env vars. The file is referenced in args like this: --args config=./path-to-config.json . We noticed that only if this file committed to the repo, it gets uploaded, the file is not native to this repo and we would ideally avoid this scenario. Thanks.

Posted one year ago
Votes Newest

Answers 2

Hi CostlyOstrich36 Thank you for reaching back and sorry for my embarrassingly long answer here.
We launch the job on a remote executor this way –
From a repository where the code lies we launch a job with clearml-task in this form: clearml-task --name projx-1-debug-$(date +%s) --project kbc --folder . --script projx/kbcrun.py --requirements custom-requirements.txt --docker python:3.9.13 --task-type training --queue aws-cpu-docker --args graph_dir=not/needed config_path=examples/transe_config_unfiltered.json
So the transe_config_unfiltered.json is the config we need to have uploaded. The engineer changes params in it before run. It’s normally a dynamic file not part of the repository. IF the file is tracked (previously added to the repo, even empty) the change is seen and is uploaded to the worker. If it’s not tracked in git, it’s not uploaded and we’re ought to have a placeholder in the repo, changing the file on every run of the job.

So it would be nice to have an option for the file to be uploaded but not be part of the repo, as the file contains dynamic parameters for launching the job. There are so many of them that putting then into --args alone is to heavyweight.

Hope that make sense.

Posted one year ago

Hi SparklingHedgehong28 , can you please elaborate on the steps you take during the process + how you connect your config to the task?

Posted one year ago
2 Answers
one year ago
one year ago
Similar posts