Previously I wasn't. I would just call model.save, but I was unsure how to do modifications in the output model, which is why I made the output model.
I hope you understood my problem statement. I want to solve the issue with or without output model. Any help would be appreciated.
Basically, at the least, would like to be able to add tags, set the name and choose to publish the model that I'm saving.
I think that what you need is to create an OutputModel , then call update weights file when you have the better model, this will also allow you to tag the model object. Would that help? Or would it make sense to use Task.models and count on the auto logging?
VexedCat68 are you manually creating the OutputModel object?
When I look at the details, model artifact in the ClearML UI, it's been saved the usual way, and no tags that I added in the OutputModel constructor are there.
Did you disable the autologging ? Are you saying the tags not appearing is a bug (it might be) ?
Also, I don't mind auto logging either if I have control over publishing the model or not directly from that script, and adding tags etc, like OutputModel.
Sure you can publish models / add tags etc, wither from the UI or programatically, with Task.models you can access the stored models, and Model(model_id) allows for tags/publish
Does that make sense ?
Also I just want to say thanks for all the help. And this tool is brilliant how it supports an end to end pipeline in this completely new space for MLOps. ...
❤ these kind words really mean a lot, thank you!
I get what you're saying. Only problem is in the case of AutoLogging, I don't have the model id, for the model being saved.
Task.models['output'] should return all the model objects the autologging created
So right now, I'm creating an OutputModel and passing the current task in the constructor. Then I just save the tensorflow keras model. When I look at the details, model artifact in the ClearML UI, it's been saved the usual way, and no tags that I added in the OutputModel constructor are there. From which to me it seems that ClearML is auto logging the model, and the model isn't connected to the OutputModel object that I created.
You're saying that the model should get connected if I call update weights file? Just a reminder, I'm saving the entire tensorflow model in saved_model format and not just the weights.
Also, I don't mind auto logging either if I have control over publishing the model or not directly from that script, and adding tags etc, like OutputModel.
Once a model is saved and published, it should be downloadable right
Well that depends if you configured CLearML to autoupload it (by default it will just log the "local location").
To auto-upload add output_uri=True to Task.Init (or specify a destination with output_uri= ` s3://bucket/ )
You can also configure it as default here:
https://github.com/allegroai/clearml/blob/65f1c0baa124efb05fb7894a5386f0dd52c0536b/docs/clearml.conf#L163
Also I just want to say thanks for all the help. And this tool is brilliant how it supports an end to end pipeline in this completely new space for MLOps. You guys have been incredibly helpful and what you've made is incredible.
They want to start integrating MLOps into the ML projects here at our company for reproducibility and continual training. ClearML popped up as a potential option so they want me to design a complete pipeline for one of our projects currently being worked on. They're hoping it fits our use case because it really is end to end and that it has support for continuous deployment built in.
Basically, right now when I save the model, it just goes in draft mode. What I want to do is that save the model only if it is better than the previous one, and once saved, publish it and have a name and tags that I want to add.
AgitatedDove14 Once a model is saved and published, it should be downloadable right? Cause I keep trying to upload a model from different projects and different tasks but it keeps overwriting the previous one, and the download option is grayed out in the UI.
As I go through the model.py file, I get what you're saying. Only problem is in the case of AutoLogging, I don't have the model id, for the model being saved.