Hi Folks! I Have An Issue With Pipeline From Functions. I'M Loading And Converting Model In One Step And Trying To Pass Pytorch Model To The Next One. How Do I Deserialize My Model.Pkl Artifact Properly In The Second Step? I Have Checked The Step Code On

Answered

Hi folks!
I have an issue with pipeline from functions. I'm loading and converting model in one step and trying to pass pytorch model to the next one. How do I deserialize my model.pkl artifact properly in the second step?

I have checked the step code on clearml webserver and I saw this line kwargs[k] = parent_task.artifacts[artifact_name].get(deserialization_function=None) . Maybe there is a way to specify deserialization_function ?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					LividReindeer58
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1635088270469632000:profile|LividReindeer58> ! parent_task.artifacts[artifact_name].get() should just work to get artifacts from the parent task (the artifact should be automatically unpickled). Are you getting any errors when you do this?

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					SmugDolphin23
				
					0

Hi @<1523701435869433856:profile|SmugDolphin23> It actually was error on my end. OpenCV linux package dependencies weren't installed correctly in nvcr.io/nvidia/pytorch:23.04-py3 docker container causing deserialization errors. After installing apt-get update && apt-get install ffmpeg libsm6 libxext6 -y everything is now auto unpickles like you said :)

  				
Posted 
	11 months ago

					More
				  		
  Report
		
					LividReindeer58
				
					0
					 × 1

Write your answer

735 Views

2 Answers

11 months ago