Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Warning:Root:Could Not Delete Task Id=6Cd7F02Be36C4361965Adf9F027Bcda5, Task Id "6Cd7F02Be36C4361965Adf9F027Bcda5" Could Not Be Found 2021-07-15 20:58:48,046 - Clearml.Task - Error - Action Failed <400/101: Tasks.Get_By_Id/V1.0 (Invalid Task Id: Id=Ff308E

WARNING:root:Could not delete Task ID=6cd7f02be36c4361965adf9f027bcda5, Task ID "6cd7f02be36c4361965adf9f027bcda5" could not be found
2021-07-15 20:58:48,046 - clearml.Task - ERROR - Action failed <400/101: tasks.get_by_id/v1.0 (Invalid task id: id=ff308eef12f141e39c68ccf7e0e0f1ff, company=d1bd92a3b039400cbafc60a7a5b1e52b)> (task=ff308eef12f141e39c68ccf7e0e0f1ff)
2021-07-15 20:58:48,046 - clearml.Task - ERROR - Failed reloading task ff308eef12f141e39c68ccf7e0e0f1ff
2021-07-15 20:58:48,055 - clearml.Task - ERROR - Action failed <400/101: tasks.get_by_id/v1.0 (Invalid task id: id=ff308eef12f141e39c68ccf7e0e0f1ff, company=d1bd92a3b039400cbafc60a7a5b1e52b)> (task=ff308eef12f141e39c68ccf7e0e0f1ff)
2021-07-15 20:58:48,055 - clearml.Task - ERROR - Failed reloading task ff308eef12f141e39c68ccf7e0e0f1ff

Hi, I have the followings errors when running the cleanup service ( https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py ). What is causing these errors?

  
  
Posted 3 years ago
Votes Newest

Answers 6


Hi GreasyPenguin14
It looks like you are trying to delete a Task that does not exist
Any chance the cleanup service is misconfigured (i.e. accessing the incorrect server) ?

  
  
Posted 3 years ago

Some of the experiments are done on a GCP instance instead of the local server on which we also run ClearML. The experiments running on GCP report to the same local clearml server, but the IP address for clearml configured on the GCP instance is different (and then forwarded). Is this the problem?

  
  
Posted 3 years ago

The odd thing it was able to authenticate but then it could not find the Task to delete.
Could it be someone already deleted the Task ?
(BTW: a new version of the cleanup service is in the working 🙂 )

  
  
Posted 3 years ago

I restarted the cleanup service. Now I get some messages like this:

2021-07-16 12:39:46,736 - clearml.storage - ERROR - Failed creating storage object file:// Reason: 'NoneType' object has no attribute 'replace'
2021-07-16 12:39:46,736 - clearml.Task - ERROR - Failed deleting None: 'NoneType' object has no attribute 'delete'
WARNING:root:Could not delete Task ID=eb11c92928af477e9e732d0cad47a57e, sequence item 0: expected str instance, NoneType found

any idea?

  
  
Posted 3 years ago

Seems like a Task contained an invalid artifact link.
I wouldn't sweat over it, it basically a warning that it could not locate the actual file to delete (albeit an ugly warning 🙂 )
I think AnxiousSeal95 would know when will the new version be ready.
regardless, is it actually deleting old Tasks ?

  
  
Posted 3 years ago

A new version should be available in a week and a half or so 😄

  
  
Posted 3 years ago