Unanswered
Hi,
We Have Clearml Version Webapp: 1.3.0-165 • Server: 1.3.0-165 • Api: 2.17
Suddenly All Experiments Were Removed.
After Checking Logs We See That Before 6 Days Elasticsearch Container Was Terminated Due To Out Of Memory And New One Was Created Instead
Thanks CostlyOstrich36
Actually I was able to find IP of the machine where API call was triggered in web logs and found the user who run the delete action.
User tried to remove only archived experiments in his project( tried several times and got some errors ) and that is what we see in API call - somehow Clearml removed all server experiments 🤔
Any idea why this might happened if user only run "delete archived experiments of his project" in WEB UI ?xx.xxx.xxx.xx - - [21/Nov/2022:17:39:14 +0000] "POST /api/v2.17/tasks.delete_many HTTP/1.1" 499 0 "
" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36" "-"
162 Views
0
Answers
one year ago
one year ago