Ok, the mistake was staring me in the eye the entire time.

I thought that set_metadata worked on the local output model. But what it actually does is try to update metadata on the remote. The output model doesn't exist on the remote before you call update_weights , so you have to set_metadata after upload.