Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I Was Wondering If Clearml Offers Any Solutions For The Following Situation. I Have Created A Queue (Say Q) For Model Training Tasks, And Two Agents Listening To It. These Two Agents Have Been Spinned Up On A Machine With Gpus. Suppose Neither Of Thes

Hi!
I was wondering if ClearML offers any solutions for the following situation. I have created a queue (say Q) for model training tasks, and two agents listening to it. These two agents have been spinned up on a machine with GPUs. Suppose neither of these agents is available due to an unintentional machine shutdown at any given time. For this kind of situation, I have an emergency agent listening to the queue Q (this agent is running on a machine where there is only CPU available). So my question, is it possible to get the emergency agent to ONLY pull tasks when the other two agents are not available? Since currently this emergency agent is also pulling tasks when the other two are busy

  
  
Posted 3 years ago
Votes Newest

Answers 13


But how could I know whether an agent is up or not? Is it from the CLI or SDK?

  
  
Posted 3 years ago

You can either use REST API or the APIClient (which can send the same request)

  
  
Posted 3 years ago

My guess is to manually read and parse the string that clearml-agent list returns, but I'm pretty sure there's a cleaner way to do it, isn't there?

  
  
Posted 3 years ago

Should be something like:
` from clearml.backend_api.session.client import APIClient

client = APIClient()

response = client.workers.get_all() `

  
  
Posted 3 years ago

I mean what should I write in a script to import the APIClient? (sorry if I'm not explaining myself properly 😅 )

  
  
Posted 3 years ago

But what is the name of that API library in order to have access to those commands from Python SDK?

  
  
Posted 3 years ago

Perfect, that's exactly what I was looking for 🙂 Thanks!

  
  
Posted 3 years ago

Just try it out manually and see what you get :)

  
  
Posted 3 years ago

Where can I find this documentation?

  
  
Posted 3 years ago

Hi GiganticTurtle0 ,
You can't set an agent to pull tasks only if other agents are off-line, but you can certainly have some monitoring script checking if the other agents are up and if not, starting this CPU agent - will that work?

  
  
Posted 3 years ago

You can call the API directly and get a list of existing agents

  
  
Posted 3 years ago

Response contains both the raw JSON response as well as an object with properties

  
  
Posted 3 years ago
1K Views
13 Answers
3 years ago
one year ago
Tags