Hi Everyone, I Am Just Wondering Whether The Bugs Regarding The Deletion Of Tasks Is Fixed In The Current Version? E.G. This Happening When You Want To Delete A Lot Of Tasks.

Answered

Hi everyone,
I am just wondering whether the bugs regarding the deletion of tasks is fixed in the current version?
E.g. this happening when you want to delete a lot of tasks.
ERROR - Task deletion failed: Action failed <500/100: tasks.delete/v1.0 (General data error (TransportError(500, 'search_phase_execution_exception', 'Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting.')))> (move_to_trash=False, force=True, task=8cb074988bec4f398096909957feaa16, delete_external_artifacts=True)

  				
Posted 
	9 months ago

					More
				  		
  Report
		
					ReassuredTiger98
				
					0
					 × 1

Votes Newest

Answers 6

Thanks a lot. Will try it!

  				
Posted 
	8 months ago

					More
				  		
  Report
		
					ReassuredTiger98
				
					0
					 × 1

@<1523701087100473344:profile|SuccessfulKoala55> Only when I delete on self-hosted.
@<1523712723274174464:profile|LazyFish41> WebApp: 1.10.0-357 • Server: 1.10.0-357 • API: 2.24

This has been happening with every version of clearml-server ever. Most probably there should be a queue in front of ES, so it does not process to many request at the same time?

  				
Posted 
	8 months ago

					More
				  		
  Report
		
					ReassuredTiger98
				
					0
					 × 1

@<1523701868901961728:profile|ReassuredTiger98> Strange:( in 1.10 we already had the code for clearing ES scrolls created during the task deletion. I would recommend upgrading to the latest release v1.12.1 anyway. In addition you can instruct ES to allow more open scrolls like below. By default it is limited to 500.

  				
Posted 
	8 months ago

					More
				  		
  Report
		
					AppetizingMouse58
				
					0

Hi @<1523701868901961728:profile|ReassuredTiger98> , this is actually an issue with ES resources being exahusted. Does it only happen when you delete? Is this a self hosted server?

  				
Posted 
	9 months ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

curl -X PUT localhost:9200/_cluster/settings -H 'Content-Type: application/json' -d'{"persistent" : {"search.max_open_scroll_context": 1000}}'

  				
Posted 
	8 months ago

					More
				  		
  Report
		
					AppetizingMouse58
				
					0

Hi @<1523701868901961728:profile|ReassuredTiger98> , what version of the apiserver are you using?

  				
Posted 
	9 months ago

					More
				  		
  Report
		
					AppetizingMouse58
				
					0

Write your answer

567 Views

6 Answers

9 months ago

8 months ago