Hi Team! Is There A Way To Make Clearml’S Aws Autoscaler And Queues Resource-Aware Please? I.E. If We Can Say, As We Enqueue Our Job, How Much Ram Or Gpu-Ram Or Even Gpus It Needs, Have The Scheduler/Autoscaler Dispatch The Job To Instances That Are Of Th

Unanswered

Thanks @<1523701070390366208:profile|CostlyOstrich36> !

I hadn’t found the multiple-resources within the same autoscaler. Could you point me to the right place please? Are they all used interexchangeably based upon availability, rather than based on job needs?
We thought of using separate queues (we do that for CPU vs GPU queues), but having ClearML automatically dispatch to the right based on a job specification would be more flexible. (for example, we could then think to dispath dynamically to the right instance type based on the hyperparameters we’re searching)
It also becomes much simpler when we start to mix single-GPU and multi-GPU jobs into the same queue.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SolidGoose91
				
					0
					 × 1

110 Views

0 Answers

one year ago