Unanswered
Hey All.
Question Regarding Scheduling And Orchestration. Does Clearml Provide Any Tooling To Schedule Entire Training Pipelines And To Trigger Training Pipelines In Response To Events, E.G. Degraded Model Performance Alerting?
Say we have a DAG running on airflow every 30 mins. The purpose of this DAG is to aggregate results of model performance. If model performance is poor, then it sends a message to a queue with some config on which model to re-train.
I would like to use a TaskScheduler to poll this queue every X interval, to check whether a training pipeline needs to be kickstarted or not
104 Views
0
Answers
2 years ago
one year ago