Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Guys, I Have 4 Workers (2 In Default And 2 In Service Queue On Same Machine) And Running A Cron Job Of Data Preparation.It Works Well For About 3 Days But After That Tasks Are Getting Failed By Their Own With Given Below Error.Can Anyone Help Me O

Hello guys, i have 4 workers (2 in default and 2 in service queue on same machine) and running a cron job of data preparation.It works well for about 3 days but after that tasks are getting failed by their own with given below error.Can anyone help me out?
image

  
  
Posted 11 months ago
Votes Newest

Answers 6


Hey @<1526734437587357696:profile|ShaggySquirrel23> , what version of the clearml-agent are you using? Also, if I were you I’d check how much free disk there’s on the machine running the agents

  
  
Posted 11 months ago

clearml version: 1.5.1
free space:12GB

  
  
Posted 11 months ago

Hello guys, i have 4 workers (2 in default and 2 in service queue on same machine)

Hi @<1526734437587357696:profile|ShaggySquirrel23>
I think what happens is one agent is deleting it's cfg file when it is done, but at least in theory each one should have it's own cfg
One last request can you try with the agent's latest RC version 1.5.3rc2 ?

  
  
Posted 11 months ago

Can you also tell what OS are you using? And when you mentioned that the clearml version: 1.5.1 did you mean the ClearML package or the clearml-agent package? Because they are different

  
  
Posted 11 months ago

can anyone help me about this?

  
  
Posted 11 months ago

clearml-agent version (output of clearml-agent --version).I am using aws ec2 linux image.

  
  
Posted 11 months ago