Hi Everyone! Each Clearml Task Has Its Own Info Tab In Ui, Which Contains Additional Information About Resources, Workers, Queues Etc. When One Restarts Task On Another Queue, Old Information Disappears (And New Information About New Queue Appears Accord

Answered

Hi everyone!

Each clearml task has its own info tab in UI, which contains additional information about resources, workers, queues etc. When one restarts task on another queue, old information disappears (and new information about new queue appears accordingly). How can we parse history of runs (info tab information for each run through restart)?

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					SweetShells3
				
					0
					 × 1

Votes Newest

Answers 2

Yes, I know, but it is possible to save history for restarted tasks?

When we use dynamic MIG allocation on GPU and meet the limits for project (or another project with higher rank runs task that requires resources already allocated by low-rank project), it stops the experiment and then run another one. The first one runs after resources are freed up. As far as I understand, it can't clone experiment, only restart :(

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					SweetShells3
				
					0
					 × 1

Hi @<1569496075083976704:profile|SweetShells3> , I would suggest cloning experiments. This way you have full history of everything

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

467 Views

2 Answers

4 months ago