Unanswered
Hi,
We Have Clearml Version Webapp: 1.3.0-165 • Server: 1.3.0-165 • Api: 2.17
Suddenly All Experiments Were Removed.
After Checking Logs We See That Before 6 Days Elasticsearch Container Was Terminated Due To Out Of Memory And New One Was Created Instead
You can restore these tasks by copying or moving them from task__trash into task collection. But the events for these tasks cannot be restored. About the user who deleted them unfortunately ClearML does not record this info in Mongo and without logging to ES there is no place to retrieve it (I can suggest using Kibana to monitor ES). You can try to inspect the mongo collection url_to_delete. It contains all the links from the deleted tasks that should be removed from the fileserver. If you see there any documents that correspond to files from the deleted tasks then the user recorded in this docs is the one who performed the delete.
164 Views
0
Answers
2 years ago
one year ago