Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello, What Happens To Running Experiments When The Clearml Agent Is Killed (And Restarted A Few Minutes Later)?

Hello, what happens to running experiments when the ClearML Agent is killed (and restarted a few minutes later)?

  
  
Posted 28 days ago
Votes Newest

Answers 3


I don;t think there is a "kill task" code. By principle, in Linux, as a parent process, ClearML agent launch the training process. When a parent process is terminated, the linux kernel will, in most of the case, kill all child processes, including your training process.
There may be some way to resume a task from ClearML agent when it restart, but I don;t think that is the default behavior

  
  
Posted 27 days ago

If the agent is the one running the experiment, very likely that your task will be killed.
And when the agent come back, immediately or later, probably nothing will happen. It won't resume ...

  
  
Posted 28 days ago

Thanks for your reply. Would you be able to point me towards a section in the documentation or maybe even a piece of code that handles the killing of the tasks?

  
  
Posted 27 days ago
70 Views
3 Answers
28 days ago
27 days ago
Tags