Some of the experiments are done on a GCP instance instead of the local server on which we also run ClearML. The experiments running on GCP report to the same local clearml server, but the IP address for clearml configured on the GCP instance is different (and then forwarded). Is this the problem?

The odd thing it was able to authenticate but then it could not find the Task to delete.
Could it be someone already deleted the Task ?
(BTW: a new version of the cleanup service is in the working 🙂 )

Hi GreasyPenguin14
It looks like you are trying to delete a Task that does not exist
Any chance the cleanup service is misconfigured (i.e. accessing the incorrect server) ?

Seems like a Task contained an invalid artifact link.
I wouldn't sweat over it, it basically a warning that it could not locate the actual file to delete (albeit an ugly warning 🙂 )
I think AnxiousSeal95 would know when will the new version be ready.
regardless, is it actually deleting old Tasks ?

A new version should be available in a week and a half or so 😄

I restarted the cleanup service. Now I get some messages like this:

2021-07-16 12:39:46,736 - clearml.storage - ERROR - Failed creating storage object file:// Reason: 'NoneType' object has no attribute 'replace'
2021-07-16 12:39:46,736 - clearml.Task - ERROR - Failed deleting None: 'NoneType' object has no attribute 'delete'
WARNING:root:Could not delete Task ID=eb11c92928af477e9e732d0cad47a57e, sequence item 0: expected str instance, NoneType found

any idea?