Does Clearml Have An Integrated Approach To Querying Model Metrics? The Objective Is To Log All Model Run Outputs In Order To Be Able To Query Them For Deployment At A Later Stage. As It Stands, I Am Currently Developing My Own Package To Log The Metrics

Answered

Does clearML have an integrated approach to querying model metrics? The objective is to log all model run outputs in order to be able to query them for deployment at a later stage. As it stands, I am currently developing my own package to log the metrics of interest & to deposit them on the cloud.

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TenseOstrich47
				
					0
					 × 1

Votes Newest

Answers 3

I think there is more complexity to what I am trying to achieve, but this will be a good start. Thanks!

I will need to log data set ID, transformer (not the NN architecture, just a data transformer), the model (with all hyperparameters & metadata) etc. and how all things link

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TenseOstrich47
				
					0
					 × 1

Well TenseOstrich47 you can certainly add any metadata to the model itself at training time (or as a post-training job if the model is not published). I'm actually doing a couple of videos on this on the ClearSHOW, tomorrow's episode should explain this approach and there will be examples next week. DM me here if you want to know more.
https://www.youtube.com/watch?v=r2BMMDzfyA0&list=PLMdIlCuMqSTkXBApOMqg2S5IeVfnkq2Mj

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					GrumpyPenguin23
				
					0
					 × 1

Hi TenseOstrich47 ,

If you want to get all the scalars. you can use task.get_last_scalar_metrics() , can this help?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TimelyPenguin76
				
					0
					 Administrator

Write your answer

899 Views

3 Answers

3 years ago

one year ago