CostlyOstrich36 and at what point the model weights will be actually uploaded?

Hmmmm, fair point.

ok, got it, thanks

I think the set_default_upload_uri is for all output models, while set_upload_destination is for a specific model/file

And can I store models with no attachment to tasks? For example, original pretrained checkpoints

but for datasets there seems to be a way to do it from the code

Ohh, like a query based only of the stored url ?
Do you also has the creating Task ?

https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel/#update_weights

AgitatedDove14 not exactly:
input: just a checkpoint file
output: a clearml model entity + stored weights on S3

Thanks. What is the difference between import_model and load_model ?

Why can I only call import_model with weights_url and not model name or ID? This means I need to call query_models first, if I got it right

All output models in the script, that is 🙂

Why can I only call 

import_model

Actually creates a new Model object in the system
InputModel(id) will "load" a model based on the model id
Make sense ?

load_model will get a link to a previously registered URL (i.e. it search a model pointing to the specific URL, if it finds it, it will get you the Model object)

And can I store models with no attachment to tasks?

Assuming you have the Model ID :
model = InputModel(model_id='aabbcc') local_file_or_folder = model.get_weights()Is this what you are looking for?

ok, got it, thanks

No, I just want to register a new model in the storage.

Is the model file is already uploaded, you can register it without a Task:
InputModel.import_model(...)https://github.com/allegroai/clearml/blob/b3a2b3425c5098ebfc0598c9dfb3e670d4a87706/clearml/model.py#L521

I need to create a separate task for this right?

If you want the model to be uploaded, then yes you have to create a Task.

No, I just want to register a new model in the storage. I need to create a separate task for this right?

what's the difference between set_default_upload_uri and set_upload_destination ?

specifying target storage for OutputModel from the config is bad idea: it should be project-specific, not agent-specific

MelancholyElk85 if you are manually adding models OutputModel, then when you call update_weights(...) upload will start in the background (if the process ends it will wait until the upload is competed). You can also specify auto_delete_file which will delete the local copy once the upload completes

How about this by the way?
https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#outputmodelset_default_upload_uri

MelancholyElk85 , you can specify the uri for that in your ~/clearml.conf file under sdk.development.default_output_uri

Please note that you don't provide target storage for InputModel since it's an input, and can be used only as an existing object in the system 🙂

CostlyOstrich36 so it's the same problem as https://clearml.slack.com/archives/CTK20V944/p1636373198353700?thread_ts=1635950908.285900&cid=CTK20V944