Hi There, I'Ve Been Trying To Work With Trains And I Wanted To Save A Folder As The Model Like When Using The "Transformers" Library. They Have This "Save_Pretrained" Method To Their Models. It Saves The Pytorch Model And You Detect It Well, But Only That

Answered

Hi there, I've been trying to work with trains and I wanted to save a folder as the model like when using the "transformers" library. They have this "save_pretrained" method to their models. It saves the pytorch model and you detect it well, but only that. Not anything else in that folder. Is there a way to override/avoid this auto output file and define it myself?

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					PompousBeetle71
				
					0
					 × 1

Votes Newest

Answers 2

Hi PompousBeetle71 , Trains will log all the torch.save call, I'm assuming they do not actually use it for the rest of the files on that folder.
If you like to share a code snippet we could see if we could auto-magically log it You could use artifacts and store the entire folder. It will zip it an upload it. Then you can reuse it from other experiments. https://allegro.ai/docs/task.html?highlight=artifact#trains.task.Task.upload_artifact
Example:
task.upload_artifact('transformer', './my_trans_folder')Then from another experiment you could do:
local_copy_of_the_folder = Task.get_task(task_id='11111').artifacts['transformer'].get()

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Yeah, I thought to use artifact, wondered if I can avoid using it or on the other hand, use only it just to define the "the model" as a folder.
Thanks.

  				
Posted 
	4 years ago

					More
				  		
  Report
		
					PompousBeetle71
				
					0
					 × 1

Write your answer

809 Views

2 Answers

4 years ago

one year ago