Hi Folks! I Have An Issue With Pipeline From Functions. I'M Loading And Converting Model In One Step And Trying To Pass Pytorch Model To The Next One. How Do I Deserialize My Model.Pkl Artifact Properly In The Second Step? I Have Checked The Step Code On

Answered

Hi folks!
I have an issue with pipeline from functions. I'm loading and converting model in one step and trying to pass pytorch model to the next one. How do I deserialize my model.pkl artifact properly in the second step?

I have checked the step code on clearml webserver and I saw this line kwargs[k] = parent_task.artifacts[artifact_name].get(deserialization_function=None) . Maybe there is a way to specify deserialization_function ?

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					LividReindeer58
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1635088270469632000:profile|LividReindeer58> ! parent_task.artifacts[artifact_name].get() should just work to get artifacts from the parent task (the artifact should be automatically unpickled). Are you getting any errors when you do this?

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					SmugDolphin23
				
					0

Hi @<1523701435869433856:profile|SmugDolphin23> It actually was error on my end. OpenCV linux package dependencies weren't installed correctly in nvcr.io/nvidia/pytorch:23.04-py3 docker container causing deserialization errors. After installing apt-get update && apt-get install ffmpeg libsm6 libxext6 -y everything is now auto unpickles like you said :)

  				
Posted 
	4 months ago

					More
				  		
  Report
		
					LividReindeer58
				
					0
					 × 1

Write your answer

285 Views

2 Answers

4 months ago