We have the following, works fine (we also use internal zip packaging for our models):
model = OutputModel(task=self.task, name=self.job_name, tags=kwargs.get('tags', self.task.get_tags()), framework=framework)
model.connect(task=self.task, name=self.job_name)
model.update_weights(weights_filename=cc_model.save())
I have already been trying to contribute (have three pull requests), but honestly I feel it is a bit weird, that I need to update a documentation about something I do not understand, while I actually try to evaluate if ClearML is the right tool for our company...
@<1523701087100473344:profile|SuccessfulKoala55> : That is the link I posted as well. But this should be mentioned also at places where it is about about the external or non-external storage. Also it should be mentioned everywhere we talk about models or artifacts etc. Not necessarily in details, but at least with a sentence and a link.
@<1523704157695905792:profile|VivaciousBadger56> I'm not sure I'm following you - is the issue not being able to upload to the ClearML server or to load the downloaded file?
FWIW, we prefer to set it in the agent’s configuration file, then it’s all automatic
@<1523704157695905792:profile|VivaciousBadger56> It seems like whatever you pickled in the zip file relies on some additional files that are not pickled.
We're certainly working hard on improving the documentation (and I do apologize for the frustrating experience)
@<1523704157695905792:profile|VivaciousBadger56> regrading: None
Is this a discussion or PR ?
(general ranting is saved for our slack channel 🙂 )
Yes, you're correct, I misread the exception.
Maybe it hasn't completed uploading? At least for Datasets one needs to explicitly wait IIRC
@<1523701083040387072:profile|UnevenDolphin73> : Thanks, but it does not mention the File Storage of "ClearML Hosted Server".
I can only say I’ve found ClearML to be very helpful, even given the documentation issue.
I think they’ve been working on upgrading it for a while, hopefully something new comes out soon.
Maybe @<1523701205467926528:profile|AgitatedDove14> has further info 🙂
The documentation is messy, I’ve complained about it the in the past too 🙈
Heh, good @<1523704157695905792:profile|VivaciousBadger56> 😁
I was just repeating what @<1523701070390366208:profile|CostlyOstrich36> suggested, credits to him
Heh, well, John wrote that in the first reply in this thread 🙂
And in Task.init
main documentation page (nowhere near the code), it says the following -
Hi all, sorry for not being so responsive today 🙏
By the way, output_uri is also documented as part of the Task.init() docstring ( None )
@<1523701083040387072:profile|UnevenDolphin73> : If I do, what should I configure how?
But, I guess @<1523701070390366208:profile|CostlyOstrich36> wrote that in a different chat, right?
@<1523701083040387072:profile|UnevenDolphin73>
@<1523701087100473344:profile|SuccessfulKoala55> Also, I think that - in this case, but also in other cases - the issue is not just the documentation, but also the design of the SDK.
@<1523701083040387072:profile|UnevenDolphin73> : I do not get this impression, because during update_weights
I get the message
2023-02-21 13:54:49,185 - clearml.model - INFO - No output storage destination defined, registering local model C:\Users..._Demodaten_FF_2023-02-21_13-53-51.624362.model
Hi @<1523704157695905792:profile|VivaciousBadger56> , you can configure Task.init(..., output_uri=True)
and this will save the models to the clearml file server
@<1523701083040387072:profile|UnevenDolphin73> : From which URL is your most recent screenshot?
FWIW It’s also listed in other places @<1523704157695905792:profile|VivaciousBadger56> , e.g. None says:
In order to make sure we also automatically upload the model snapshot (instead of saving its local path), we need to pass a storage location for the model files to be uploaded to.
For example, upload all snapshots to an S3 bucket…
@<1523701083040387072:profile|UnevenDolphin73> : I do not see any way to download the model manually from the web app either. All I see is the link to the file on my harddrive (see shreenshot).
The second process says there is not file at all. I think, all that happened is that the update_weights
only uploaded the location of the .zip
file (which we denote as a .model
file) on my harddrive, but not the file itself.
It should store it on the fileserver, perhaps you're missing a configuration option somewhere?