Hi UnsightlyShark53 ,
If you want to save anything from your experiment, you can use the artifacts, you can view this example - https://github.com/allegroai/trains/blob/master/examples/artifacts_toy.py
Can this do the trick?
Hi, sorry for late reply! Yes that could work in a way, though I think only saving to the model snapshot feature would have been great!
Oh sorry, I forgot to specify one thing. I only really want to save once (because the model is large), so I hoped the model would only be saved to the Trains URI. This was why I was wondering if there was an explicit way to call trains to save, so I don't also save at the location given in torch.save(...). Is this possible?
I see, I will try this code snippet 🙂
Hi! So if I understand the documentation correctly, then TRAINS support saving snapshots to some uri. Just to test it for now, I just want it to save the snapshots locally into trains_storage with the file structure given in the documentation is referenced previously.
I think the first thing I should ask is just how do I save a pytorch snapshot with trains? Can it be done explicitly with a method, or is it done implicitly by TRAINS somehow picking up that I called torch.save(...)?
What I hope to achieve is: Save a snapshot of my model for each epoch using TRAINS to the given uri
Hi UnsightlyShark53 ,
Trying to understand the scenario, so you want the model to be saved in trains_storage
dir but trains
saves it in trains_storage/trains_storage
? Or the torch.save
doesn't save in the path?
Trains do patch the torch save function 🙂
If you like, you can save the model for each epoch by having a unique name for it. The model will be saved in the output_uri
path you have in the Task.init
command.
For example, this code will save a model for every epoch:
for epoch in range(num_of_epoch): # Create a model torch.save(model, "model_epoch_{}".format(epoch))