Hi MelancholyBeetle72 , that's a very interesting case. I can totally understand how storing a model and then immediately renaming it breaks the upload. A few questions, is there a way for pytorch lightning not to rename the model? Also I wonder if this scenario happens a lot (storing model and changing it) . I think the best solution is for Trains to create a copy of the file and upload it in the background. That said the name will still end with .part What do you think?
MelancholyBeetle72 it will be great if you could also open an issue on Trains and reference the pytorch lightning issue, could you please?
is there a way for pytorch lightning not to rename the model?
I think no, since pytorch-lightning ensures atomarity of save operation by this logic.
I think the best solution is for Trains to create a copy of the file and upload it in the background
Yes, sounds good.
Yeah, I'll open an issue on Trains Github
MelancholyBeetle72 thanks! I'll see if we could release an RC with a fix soon, for you to test :)
MelancholyBeetle72 there is an RC with a fix, check the GitHub issue for details :)
Wow, astonishingly fast 🙂
Ok, I'll check my case and leave feedback