ReassuredTiger98 Strange:( in 1.10 we already had the code for clearing ES scrolls created during the task deletion. I would recommend upgrading to the latest release v1.12.1 anyway. In addition you can instruct ES to allow more open scrolls like below. By default it is limited to 500.
curl -X PUT localhost:9200/_cluster/settings -H 'Content-Type: application/json' -d'{"persistent" : {"search.max_open_scroll_context": 1000}}'
Hi ReassuredTiger98 , this is actually an issue with ES resources being exahusted. Does it only happen when you delete? Is this a self hosted server?
SuccessfulKoala55 Only when I delete on self-hosted.
LazyFish41 WebApp: 1.10.0-357 • Server: 1.10.0-357 • API: 2.24
This has been happening with every version of clearml-server ever. Most probably there should be a queue in front of ES, so it does not process to many request at the same time?
Hi ReassuredTiger98 , what version of the apiserver are you using?