Hey, I’M Getting A Lot Of These

Answered

Hey,
I’m getting a lot of these warnings that are always followed by my job dying…
[clearml.Task] WARNING : ### TASK STOPPED - USER ABORTED - STATUS CHANGED ###What causes this warning to be printed? Can this cause the program to crash in weird ways? The correlation is quite high in our setup. Can’t really tell if causation exists…

A sample scenario where this would happen:
A single program where we sequentially create a task and run a job. We do call task.mark_completed() and task.close() after the first task.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CourageousKoala93
				
					0
					 × 1

Votes Newest

Answers 6

CourageousKoala93 thanks, task.close() solved for me

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					RoughHedgehog31
				
					0
					 × 1

CourageousKoala93 can you try just calling Task.close() and seeing if that works (i.e. solves the issue)?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

AgitatedDove14 make sense.

SuccessfulKoala55 sdk version 1.4.1rc.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CourageousKoala93
				
					0
					 × 1

CourageousKoala93 when you call Task.close() it will mark the task as completed, there is no need to do that manually. The idea with mark_completed is that you can forcefully change the state if needed, or externally stop the task and mark it completed. Make sense?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Hi CourageousKoala93 , what clearml sdk version are you using?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Is there a way to clean up clearml completely?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CourageousKoala93
				
					0
					 × 1

Write your answer

1K Views

6 Answers

2 years ago

one year ago