Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello! I'M Running Clearml-Server On Kubernetes, And It Seems My Models Are Not Really Saved. I See That Doing Task.Init(Output_Uri=True) Should Send Models To Fileserver. The Models Are Visible In The Ui But The Download Button Is Greyed Out And When I D

Hello! i'm running clearml-server on kubernetes, and it seems my models are not really saved. i see that doing task.init(output_uri=True) should send models to fileserver. The models are visible in the UI but the download button is greyed out and when i do .get_local_copy() i get None

  
  
Posted one year ago
Votes Newest

Answers 30


AntsyElk37

and when i try to use --output-uri i can't pass true because obviously i can't pass a boolean only strings

hmm, that sounds right, I think we should fix that so when using --output-uri true the value that is passed is actually True, not the string "true".
Regrading the issue itself:
are you saying --skip-task-init is being ignored ? and it always adds the Task.init call? you can also pass --output-uri https://files.clear.ml (which is the same as True) , does that work?

  
  
Posted one year ago

...I'm not sure I follow, the clearml-task is designed to always be used so that at the end the agent will be running the Task. What am I missing?

  
  
Posted one year ago

no i don't think so, i think rather Task.init is only used for running outside of agent

  
  
Posted one year ago

it seems that whatever i pass to Task.init is ignored

  
  
Posted one year ago

i can pass any crazy value i want.. it doesn't matter. however, i can use --output_uri= s3://blabla and then at least i get the error that it cannot use that bucket

  
  
Posted one year ago

so it seems that it takes output_uri from the clearml commandline but not from the Task.init inside the scripot

  
  
Posted one year ago

(same for environment variable)

  
  
Posted one year ago

and ... clearml-agent takes a --project and a --name argument that are mandatory , so these are never taken from Task.init

  
  
Posted one year ago

hello, i'm still not able to save clearml models. They are generated and registered okay, but they are not on the fileserver. i now have Task.init(output_uri=True) and i also have --skip-task-init in clearml commandline so that it doesn't overwrite the task.init call

  
  
Posted one year ago

task = Task.init(project_name='examples', task_name='moemwap', output_uri=True, reuse_last_task_id=False)

  
  
Posted one year ago

this is my cmdline: clearml-task --name hla --requirements requirements.txt --project examples --output-uri http://clearml-fileserver:8081 --queue aws-instances --script keras_tensorboard.py

  
  
Posted one year ago

and if you add --skip-task-init ?
I think what happens is that the clearml-Task, adds a Task.init call without the output_uri that is called before "your" Task.init, and this is what causes it to be ignored. Could that be the case?

  
  
Posted one year ago

this is the output of the training. it doens't try to upload (note that this is my second try so it already found a model with that name, but on my first try it didn't work either)

  
  
Posted one year ago

well it made a difference (the code for the init() is not added anymore) but it still didn't take my output uri

  
  
Posted one year ago

it didn''t make a difference

  
  
Posted one year ago

it was to test if reuse_last_task_id made any effect (i have the impression it doesn't)

  
  
Posted one year ago

and when i try to use --output-uri i can't pass true because obviously i can't pass a boolean only strings

  
  
Posted one year ago

but i still think the same should be possible using the Task.init

  
  
Posted one year ago

don't know.. but i see for instance when using clearml-task i can put any (even nonsensical) values in Task.init

  
  
Posted one year ago

AgitatedDove14 your trick seems to work (i had to change the url to reflect the fact i run on k8s)

  
  
Posted one year ago

and also, on the tutorials that do something with task.init, the example always talks about running locally and not in the agent

  
  
Posted one year ago

this is now in my python script:

  
  
Posted one year ago

this is the script shown by clearML ui. so the task.init call looks right

  
  
Posted one year ago

the model has this information ... the /tmp seem local URIs suggesting that it doesn't even try to upload them

  
  
Posted one year ago

but i still think the same should be possible using the Task.init

This is the part the I find confusing:
Task.init(..., output_uri=True) is working for me, what is that setup that caused this line to "fail"?

  
  
Posted one year ago

i set reuse_last_task_id to false to force creation of a new task in all cases

  
  
Posted one year ago

well it doesn't fail. but whatever i set gets ignored

  
  
Posted one year ago

i added --skip-task-init too

  
  
Posted one year ago

for comparison: this is when i use --output-uri

  
  
Posted one year ago

its as if the line is not there

  
  
Posted one year ago
195 Views
30 Answers
one year ago
8 months ago
Tags