Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I'M Training A Tensorflow Model And Saving It In The End. I Looked At The Outputmodel Class. How Do I Connect The Model I'M Saving To The Outputmodel?

I'm training a tensorflow model and saving it in the end. I looked at the OutputModel class. How do I connect the model I'm saving to the OutputModel?

  
  
Posted 3 years ago
Votes Newest

Answers 12


Basically, right now when I save the model, it just goes in draft mode. What I want to do is that save the model only if it is better than the previous one, and once saved, publish it and have a name and tags that I want to add.

  
  
Posted 3 years ago

Previously I wasn't. I would just call model.save, but I was unsure how to do modifications in the output model, which is why I made the output model.

I hope you understood my problem statement. I want to solve the issue with or without output model. Any help would be appreciated.

Basically, at the least, would like to be able to add tags, set the name and choose to publish the model that I'm saving.

  
  
Posted 3 years ago

I think that what you need is to create an OutputModel , then call update weights file when you have the better model, this will also allow you to tag the model object. Would that help? Or would it make sense to use Task.models and count on the auto logging?

  
  
Posted 3 years ago

Also I just want to say thanks for all the help. And this tool is brilliant how it supports an end to end pipeline in this completely new space for MLOps. You guys have been incredibly helpful and what you've made is incredible.

They want to start integrating MLOps into the ML projects here at our company for reproducibility and continual training. ClearML popped up as a potential option so they want me to design a complete pipeline for one of our projects currently being worked on. They're hoping it fits our use case because it really is end to end and that it has support for continuous deployment built in.

  
  
Posted 3 years ago

So right now, I'm creating an OutputModel and passing the current task in the constructor. Then I just save the tensorflow keras model. When I look at the details, model artifact in the ClearML UI, it's been saved the usual way, and no tags that I added in the OutputModel constructor are there. From which to me it seems that ClearML is auto logging the model, and the model isn't connected to the OutputModel object that I created.

You're saying that the model should get connected if I call update weights file? Just a reminder, I'm saving the entire tensorflow model in saved_model format and not just the weights.

Also, I don't mind auto logging either if I have control over publishing the model or not directly from that script, and adding tags etc, like OutputModel.

  
  
Posted 3 years ago

When I look at the details, model artifact in the ClearML UI, it's been saved the usual way, and no tags that I added in the OutputModel constructor are there.

Did you disable the autologging ? Are you saying the tags not appearing is a bug (it might be) ?

Also, I don't mind auto logging either if I have control over publishing the model or not directly from that script, and adding tags etc, like OutputModel.

Sure you can publish models / add tags etc, wither from the UI or programatically, with Task.models you can access the stored models, and Model(model_id) allows for tags/publish
Does that make sense ?

Also I just want to say thanks for all the help. And this tool is brilliant how it supports an end to end pipeline in this completely new space for MLOps. ...

❤ these kind words really mean a lot, thank you!

  
  
Posted 3 years ago

AgitatedDove14 Once a model is saved and published, it should be downloadable right? Cause I keep trying to upload a model from different projects and different tasks but it keeps overwriting the previous one, and the download option is grayed out in the UI.

  
  
Posted 3 years ago

As I go through the model.py file, I get what you're saying. Only problem is in the case of AutoLogging, I don't have the model id, for the model being saved.

  
  
Posted 3 years ago

I get what you're saying. Only problem is in the case of AutoLogging, I don't have the model id, for the model being saved.

Task.models['output'] should return all the model objects the autologging created

  
  
Posted 3 years ago

I'll give that a try, thank you.

  
  
Posted 3 years ago

VexedCat68 are you manually creating the OutputModel object?

  
  
Posted 3 years ago

Once a model is saved and published, it should be downloadable right

Well that depends if you configured CLearML to autoupload it (by default it will just log the "local location").
To auto-upload add output_uri=True to Task.Init (or specify a destination with output_uri= ` s3://bucket/ )
You can also configure it as default here:
https://github.com/allegroai/clearml/blob/65f1c0baa124efb05fb7894a5386f0dd52c0536b/docs/clearml.conf#L163

  
  
Posted 3 years ago
1K Views
12 Answers
3 years ago
one year ago
Tags