Hi AgitatedDove14
I have multiple models for a classification task but over the time performance is degrading. I want to trigger a retrain task when F1 score drops below certain value. As part of the retraining I have to train all the models and then have to choose best one and deploy it. How do I achieve this using trains?

For classification it's F1 score but for other task it maybe and I don't think that's problem. we just have to log it right?

Correct 🙂

Give me few days, I will work on your sugestions and then let you know if I am not able to do this

Sounds good!
BTW:
previous_tasks = Task.get_tasks(task_filter={'tags': 'best'}) local_model_file = previous_tasks[0].artifcats['my_model'].get_local_copy()

Hi FancyChicken53
This is a noble cause you are after 😉
Could you be more specific on what you had in mind, I'll try to find the best example once I have more understanding ...

Questions

I want to trigger a retrain task when F1

That means that in inference you are reporting the F1 score, correct?

As part of the retraining I have to train all the models and then have to choose best one and deploy it

Are you using passing output_uri to Task.init? are you storing the model as artifact?
You can tag your model/task with "best" tag (and untag the previous one). Then in production , look for the "best" task and get its model
Thoughts?

That means that in inference you are reporting the F1 score, correct?

For classification it's F1 score but for other task it maybe and I don't think that's problem. we just have to log it right?

Are you using passing output_uri to Task.init?

No

are you storing the model as artifact?

Yes

Also I do get new data everyday for classification task, how do I tune my model everyday on that new data?

Give me few days, I will work on your sugestions and then let you know if I am not able to do this.