Hey All. Question Regarding Scheduling And Orchestration. Does Clearml Provide Any Tooling To Schedule Entire Training Pipelines And To Trigger Training Pipelines In Response To Events, E.G. Degraded Model Performance Alerting?

Unanswered

Say we have a DAG running on airflow every 30 mins. The purpose of this DAG is to aggregate results of model performance. If model performance is poor, then it sends a message to a queue with some config on which model to re-train.

I would like to use a TaskScheduler to poll this queue every X interval, to check whether a training pipeline needs to be kickstarted or not

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					TenseOstrich47
				
					0
					 × 1

175 Views

0 Answers

2 years ago

one year ago