Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
My Data Processing Scripts Are Run In The Cloud With The Help Of Clearml Autoscaler. The Cloud Doesn'T (And Won'T) Have Access To Git, Which Is In Our Internal Network. So I'M Left With Using

My data processing scripts are run in the cloud with the help of ClearML autoscaler. The cloud doesn't (and won't) have access to Git, which is in our internal network. So I'm left with using Task.force_store_standalone_script() . But I will soon have multiple similar scripts in my monorepo for different tasks. There is a lot of common code in those tasks, which I don't want to duplicate. I want to extract the code to some common module, but this common module won't be uploaded to clearml server. Right now I see only one solution: before running the script, run a tool which combines 2+ modules into single one. Anybody can think of alternative solutions?

  
  
Posted 13 days ago
Votes Newest

Answers 5


it seems that connecting it as config is more convenient than uploading an artifact, because artifacts are deleted when cloning a task. Code is very simple:

  
  
Posted 10 days ago

    common_module = task.connect_configuration("../common.py", "common.py")
    if not task.running_locally():
        import shutil
        shutil.copy(common_module, "common.py")

    from common import test_common
    test_common()
  
  
Posted 10 days ago

I also cannot create a package out of common code, because the package registry is inside the internal network as well

  
  
Posted 13 days ago

That sounds like an interesting hack 😃 I'll try it out, thanks!

  
  
Posted 11 days ago

Can you perhaps upload them as artifacts to the task and extract/import them in your script?

  
  
Posted 11 days ago
282 Views
5 Answers
13 days ago
10 days ago
Tags