Ah, thanks a lot. So for example the CleanUp Service ( https://github.com/allegroai/clearml/blob/master/examples/services/cleanup/cleanup_service.py ) should have no troubles deleting the artifacts.
Any idea why deletion of artifacts on my second fileserver does not work?
fileserver_datasets: networks: - backend - frontend command: - fileserver container_name: clearml-fileserver-datasets image: allegroai/clearml:latest restart: unless-stopped volumes: - /opt/clearml/logs:/var/log/clearml - /opt/clearml/data/fileserver-datasets:/mnt/fileserver - /opt/clearml/config:/opt/clearml/config ports: - "8082:8081"
ClearML successfully stores artifacts on there, I can download them, but when I try to delete I get
.... not deleted as error message, but 100% deletion and they get removed from the webserver, but remain on disk.
I meant that in the WebUI deletion should only be allowed for artifacts for which deletion actually works.
For example I now have a lot of lingering artifacts that exist on the fileservers, but not on the clearml-api-server (I think).
Another example: I delete a task via WebUI. ClearML-server tries to delete the task and the artifacts belonging to the task. However, it will show that the task has been successfully deleted but some artifacts have not. Now there is no way anymore to delete the artifacts except for asking the admin to manually look for files on the fileserver.
In my opinion the task should only be deleted after the artifacts have been deleted successfully . And the artifacts should only be deleted from the server when the fileserver successfully deletes the files behind the artifacts.
I fully understand, the thing is it's not the server deleting the files right now, but the UI - once we add this option in the server, we will be able to provide this flow. Still, I'm sure that it would still be a viable option to delete a task even if the associated artifacts can't be deleted for some reason (for example, what if you have an artifact weighing a few KBs that's somehow corrupted?)
Yea, when the server handles the deletes everythings fine and imo, that is how it should always have been.
I don't think it is a viable option. You are looking at the best case, but I think you should expect the worst from the users 🙂 Also I would rather know there is a problem and have some clutter than to hide it and never be able to fix it because I cannot identify which artifacts are still in use without spending a lot of time comparing artifact IDs.