ClearML FAQ | Hi There, As A Last Step Of The Model Training Pipeline, I Upload It To Clearml And Set The

Answered

Hi There, As A Last Step Of The Model Training Pipeline, I Upload It To Clearml And Set The

Hi there,
As a last step of the model training pipeline, I upload it to ClearML and set the auto_delete_file

filepath = f'models/{model_id}/model.sav' joblib.dump(model, filepath, compress=True) task.update_output_model(model_path=filepath, name='model', auto_delete_file=True)
But when I try to query it using get_local_copy() or get_weights() I get a local path of the model.
So, the model was not uploaded? It saves only the local path?

Can't I store it on ClearML server? like done with images, dataframes, etc.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					IrritableGiraffe81
				
					0
					 × 1

Votes Newest

Answers 3

Hi IrritableGiraffe81 ,

Please refer to this:
https://clear.ml/docs/latest/docs/references/sdk/task#update_output_model

First, ClearML uploads the file to the preconfigured output destination (see the Task’s

output.destination

property or call the

setup_upload

method)

I think you need to provide output_destination=True in your Task.init()

  				
Posted 
	one year ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

This is not a valid parameter: https://clear.ml/docs/latest/docs/references/sdk/task#taskinit

Also I did not find any usage example of the setup_upload method

Thanks anyway

  				
Posted 
	one year ago

					More
				  		
  Report
		
					IrritableGiraffe81
				
					0
					 × 1

IrritableGiraffe81 I think CostlyOstrich36 meant output_uri

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Write your answer

639 Views

3 Answers

one year ago

one year ago

Tags

Similar posts

Hi :slightly_smiling_face: we have a clearml pipeline that has a step that runs a multi gpu training (with hugginface), we need to invoke it with `accelerate launch` so we use `subprocess.run` inside the step but it hangs when finished. Is this the righ

Hi There. In A Clearml Pipeline Step With Docker, I Specify The Git Repo And Branch I Want To Use. How Can I Also Specify A Repos Optional Dependecies? It Uses Poetry For Deendency Management

Hello All! Is It Possible To Utilize Shared Memory In Clearml For Tasks Like Model Inference, Where Instead Of Transferring Images Over The Network (E.G., Http, Rpc), We Use A Shared Memory Extension? Please Refer To The Link Below:

Hi There, I Am Running Urbansounds Example, The First Step Is Download The Data In My Agent Machine Which Has Been Successed, But The Second Step "Preprocessing" Failed, Can Anyone Help? Thanks A Log, The Error Message: Traceback (Most Recent Call Last)

Hey Guys! I Would Love To Know How To Integrate Hpo Inside Clearml Pipelines. I Have Made A Continuous Learning Pipeline With Data Etl And Model Training And As A Next Step, It Would Be Cool To Add Hpo. Most Of The Examples On The Website Create A New Ta

Hello Everyone, I'M Currently Trying Clearml-Serving To Serve A Model Via An Endpoint. I Followed The Tutorial In The Documentation, But When I Try A Request, I Get An Error. Here It Is: Curl -X Post "

Hello Everyone, I'M Currently Trying Clearml-Serving To Serve A Model Via An Endpoint. I Followed The Tutorial In The Documentation, But When I Try A Request, I Get An Error. Here It Is: Curl -X Post "

Please Tell Me How To Find Out From A Model Whether It Is Published Or Not? I Didn'T Find The

A Question On Pipeline Artifacts And Their Use In Steps. I Want To Add An Artifact To The Pipeline (See Below), And Use It As Input To A Function Step. But The Following Does Not Work Because

Hi All. I'M Setting Up An Model Export Script That Will Export Trained Models For Edge Deployment. I Initially Thought About Setting It Up As A Trigger Scheduler, And To Have It Trigger On Tags On A Published Model, But As Time Goes By The Trigger Schedul