Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! Thanks For This Awesome Tool

Hi! Thanks for this awesome tool 😄
I wanted to know if there was a way of specifying the output_uri depending on whether I'm running on GCP or in my local server.
Because it seems that I can create the task in my server and then when I re run it in GCP, the output uri is the same used in the server
if running_on_gcp(): output_uri = "gs://..." else: output_uri = os.environ.get('HOME', '/data')

  
  
Posted 3 years ago
Votes Newest

Answers 30


is there a way to prevent from creating a new setup in my worker each time?

  
  
Posted 3 years ago

and just reuse it

  
  
Posted 3 years ago

since it didn't change (the packages installed)

  
  
Posted 3 years ago

mmm..I'm having the same issue:
in my GCP agent:

base) root@gst-cv-glema3-final-tf2-cu101:~# clearml-agent daemon --queue redness
Current configuration (clearml_agent v0.17.1, location: /root/clearml.conf):


sdk.storage.cache.default_base_dir = ~/.clearml/cache
sdk.development.default_output_uri =

  
  
Posted 3 years ago

but it still saves using the output_uri from the server that created the task

  
  
Posted 3 years ago

and in the script I removed the output_uri= , in the task initialization

  
  
Posted 3 years ago

if I launch the same script in GCP, (I don't run it as a clearml-agent), then everything works fine

  
  
Posted 3 years ago

Do you have anything in the script itself related to output_uri ?

  
  
Posted 3 years ago

no, only in the clearml.conf file
Now I removed the output_uri in the conf file of the machine that started the task, and when I run it as agent in GCP it works.
Is this a bug?

  
  
Posted 3 years ago

if I put ~/clearml in the default_output_uri key, and start the task, when run as agent in GCP I get clearml.Task - INFO - Completed model upload to file:///$github_proj_directory/~/clearml/$proj_name/$experiment_name

  
  
Posted 3 years ago

I'm not sure I understand - can you share the different settings you used in you clearml.con file (just put the relevant parts of the file here, if you can) and explain where should each setting point to?

  
  
Posted 3 years ago

My setting is the following:
-run script in tl2 (local server)
-clone task and enque it, run it in GCP
Ideally I would like to:
if the script in run in tl2 it should save to local filesystem if the script is run in GCP it should save to GS

  
  
Posted 3 years ago

Can I see the exact lines you added to the .conf file?

  
  
Posted 3 years ago

now I have:
in tl2: default_ output_uri : "~/clearml"
in GCP: default_ output_uri : "gs://..."

  
  
Posted 3 years ago

if I write:
in tl2 conf : default_ output_uri : "/home/tglema/clearml"
in GCP it saves in that same dir

  
  
Posted 3 years ago

even though I have set the default_output_uri in GCPs conf file

  
  
Posted 3 years ago

is that clearer?

  
  
Posted 3 years ago

I'm not sure...

in GCP it saves in that same dir
even though I have set the default_output_uri in GCPs conf file

You mean that in GCP it stores in "/home/tglema/clearml" even though you've set
default_ output_uri : "gs://..." in the clearml.conf file on the GCP instance?

  
  
Posted 3 years ago

How exactly did you add it to the cleaml.conf file?

  
  
Posted 3 years ago

Can I see the whole config file? (you can hide any sensitive info there, of course)

  
  
Posted 3 years ago

sure 🙂

  
  
Posted 3 years ago

this is the one in GCP

  
  
Posted 3 years ago

this is the one in tl2

  
  
Posted 3 years ago

so with these two configurations, and no output_uri in the task creation in the script:
I get model saved model in tl2 and in GCP (when run as agent):
/home/tglema/git_repo/~/clearml/

  
  
Posted 3 years ago

let me run the model_upload example in your repo instead of my script

  
  
Posted 3 years ago

ok, ran the script, and had the same issue..I detected another bug I think..going to put it outside the thread

  
  
Posted 3 years ago

do I create an issue for this as well? SuccessfulKoala55 ?

  
  
Posted 3 years ago

Please do 🙂

  
  
Posted 3 years ago