Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
My Data Processing Scripts Are Run In The Cloud With The Help Of Clearml Autoscaler. The Cloud Doesn'T (And Won'T) Have Access To Git, Which Is In Our Internal Network. So I'M Left With Using

My data processing scripts are run in the cloud with the help of ClearML autoscaler. The cloud doesn't (and won't) have access to Git, which is in our internal network. So I'm left with using Task.force_store_standalone_script() . But I will soon have multiple similar scripts in my monorepo for different tasks. There is a lot of common code in those tasks, which I don't want to duplicate. I want to extract the code to some common module, but this common module won't be uploaded to clearml server. Right now I see only one solution: before running the script, run a tool which combines 2+ modules into single one. Anybody can think of alternative solutions?

  
  
Posted 6 months ago
Votes Newest

Answers 5


Can you perhaps upload them as artifacts to the task and extract/import them in your script?

  
  
Posted 6 months ago

That sounds like an interesting hack 😃 I'll try it out, thanks!

  
  
Posted 6 months ago

    common_module = task.connect_configuration("../common.py", "common.py")
    if not task.running_locally():
        import shutil
        shutil.copy(common_module, "common.py")

    from common import test_common
    test_common()
  
  
Posted 6 months ago

it seems that connecting it as config is more convenient than uploading an artifact, because artifacts are deleted when cloning a task. Code is very simple:

  
  
Posted 6 months ago

I also cannot create a package out of common code, because the package registry is inside the internal network as well

  
  
Posted 6 months ago
881 Views
5 Answers
6 months ago
6 months ago
Tags