Hi AgitatedDove14
I have multiple models for a classification task but over the time performance is degrading. I want to trigger a retrain task when F1 score drops below certain value. As part of the retraining I have to train all the models and then have to choose best one and deploy it. How do I achieve this using trains?

Also I do get new data everyday for classification task, how do I tune my model everyday on that new data?

Questions

I want to trigger a retrain task when F1

That means that in inference you are reporting the F1 score, correct?

As part of the retraining I have to train all the models and then have to choose best one and deploy it

Are you using passing output_uri to Task.init? are you storing the model as artifact?
You can tag your model/task with "best" tag (and untag the previous one). Then in production , look for the "best" task and get its model
Thoughts?

For classification it's F1 score but for other task it maybe and I don't think that's problem. we just have to log it right?

Correct 🙂

Give me few days, I will work on your sugestions and then let you know if I am not able to do this

Sounds good!
BTW:
previous_tasks = Task.get_tasks(task_filter={'tags': 'best'}) local_model_file = previous_tasks[0].artifcats['my_model'].get_local_copy()

Hi FancyChicken53
This is a noble cause you are after 😉
Could you be more specific on what you had in mind, I'll try to find the best example once I have more understanding ...

Give me few days, I will work on your sugestions and then let you know if I am not able to do this.

That means that in inference you are reporting the F1 score, correct?

For classification it's F1 score but for other task it maybe and I don't think that's problem. we just have to log it right?

Are you using passing output_uri to Task.init?

No

are you storing the model as artifact?

Yes