Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
My Data Processing Scripts Are Run In The Cloud With The Help Of Clearml Autoscaler. The Cloud Doesn'T (And Won'T) Have Access To Git, Which Is In Our Internal Network. So I'M Left With Using

My data processing scripts are run in the cloud with the help of ClearML autoscaler. The cloud doesn't (and won't) have access to Git, which is in our internal network. So I'm left with using Task.force_store_standalone_script() . But I will soon have multiple similar scripts in my monorepo for different tasks. There is a lot of common code in those tasks, which I don't want to duplicate. I want to extract the code to some common module, but this common module won't be uploaded to clearml server. Right now I see only one solution: before running the script, run a tool which combines 2+ modules into single one. Anybody can think of alternative solutions?

Posted one month ago
Votes Newest

Answers 5

    common_module = task.connect_configuration("../common.py", "common.py")
    if not task.running_locally():
        import shutil
        shutil.copy(common_module, "common.py")

    from common import test_common
Posted one month ago

it seems that connecting it as config is more convenient than uploading an artifact, because artifacts are deleted when cloning a task. Code is very simple:

Posted one month ago

Can you perhaps upload them as artifacts to the task and extract/import them in your script?

Posted one month ago

That sounds like an interesting hack 😃 I'll try it out, thanks!

Posted one month ago

I also cannot create a package out of common code, because the package registry is inside the internal network as well

Posted one month ago
5 Answers
one month ago
one month ago