Hi MelancholyBeetle72 , that's a very interesting case. I can totally understand how storing a model and then immediately renaming it breaks the upload. A few questions, is there a way for pytorch lightning not to rename the model? Also I wonder if this scenario happens a lot (storing model and changing it) . I think the best solution is for Trains to create a copy of the file and upload it in the background. That said the name will still end with .part What do you think?
is there a way for pytorch lightning not to rename the model?
I think no, since pytorch-lightning ensures atomarity of save operation by this logic.
I think the best solution is for Trains to create a copy of the file and upload it in the background
Yes, sounds good.
Yeah, I'll open an issue on Trains Github
MelancholyBeetle72 there is an RC with a fix, check the GitHub issue for details :)
Wow, astonishingly fast 🙂
Ok, I'll check my case and leave feedback
MelancholyBeetle72 thanks! I'll see if we could release an RC with a fix soon, for you to test :)
MelancholyBeetle72 it will be great if you could also open an issue on Trains and reference the pytorch lightning issue, could you please?