Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Another Strange Behavior Of The Python Sdk Cli: After Executing Python My_Task.Py, Where My_Task.Py Creates And Send To The Queue An Experiment, The Command Returns But After Some Time Some Messages Are Printed In The Console, Such As

Another strange behavior of the python SDK CLI: after executing python my_task.py, where my_task.py creates and send to the queue an experiment, the command returns but after some time some messages are printed in the console, such as
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-startI believe a process is still running in the background. Is it expected? (v0.17.4)

  
  
Posted 3 years ago
Votes Newest

Answers 9


AgitatedDove14 ok, but this happens in my local machine, not in the agent

  
  
Posted 3 years ago

ok, but this happens in my local machine, not in the agent

resource monitoring is always running in the background, even on local machines. (of course you can turn it off)

  
  
Posted 3 years ago

yes, so it does exist the local process (at least, the command returns),

What do you mean the command returns ? are running the scipt from bash and it returns to bash ?

  
  
Posted 3 years ago

yes, so it does exit the local process (at least, the command returns), but another process is still running on the background and is logging things from time to time (such as:)
ClearML Monitor: Could not detect iteration reporting, falling back to iterations as seconds-from-start

  
  
Posted 3 years ago

yes, exactly: I run python my_script.py , the script executes, creates the task, calls task.remote_execute(exit_process=True) and returns to bash. Then, in the bash console, after some time, I see some messages being logged from clearml

  
  
Posted 3 years ago

hen, in the bash console, after some time, I see some messages being logged from clearml

JitteryCoyote63 Hmm that is strange, let me check something

  
  
Posted 3 years ago

I would expect that after calling Task.enqueue(exit=True), the local task is closed and no processes related to it is running

Ohh my apologies, I did not understand that.
Are you saying that locally you call task.remote_execute(exit_process=True) and it does not leave the local process ?

  
  
Posted 3 years ago

but if the task is now running on an agent, isn’t is possible source of conflict? I would expect that after calling Task.enqueue(exit=True), the local task is closed and no processes related to it is running

  
  
Posted 3 years ago

I believe a process is still running in the background. Is it expected? (v0.17.4)

Yes it is expected.
Basically it reports that the resource monitoring did not detect any "iterations"/"steps" reporting, so instead of reporting resources based on iterations it reports based on time. Make sense ?

  
  
Posted 3 years ago
1K Views
9 Answers
3 years ago
one year ago
Tags