From a pipeline, you can use PipelineController.upload_model( name , path ) and specify in path the path you used to save your model from your script.

It depend if you have added the parameter monitor_models=['mymodel'] in the PipelineDecorator.component. If not, the model will not be logged. If you have, then torch.save will save your model to a local path, that will be logged into the main pipeline task

Also, does PipelineDecorator.upload_model store anything on the fileserver ? I can't seem to understand the use of PipelineDecorator.upload_model() apart from making a model appear on the pipeline task

Basically I have a script that generates a pipeline report in pdf format, was wondering if that can be logged

Are Kwargs supported in functions decorated as a pipeline component?

They are, but I think the main issue is the casting, without prior knowledge, everything will be a tring

Hey, is it possible for me to upload a pdf as an artefact?

Sure, just point to the file and it will upload it for you 🙂

I'm asking this because my kwargs is observed as an empty dict if printed

Hey, is it possible for me to upload a pdf as an artefact?

Hey David , I was able to get things uploaded to the fileserver by a change in the conf

Hi WickedElephant66

So I'm trying to upload an artefact to clearml’s fileserver(I have a self hosted clearml server running),

Are you trying to upload an artifact? If so I would do:
task.upload_artifact('local file', artifact_object="/path/to/file")Or is it about Model files?
You can alst check how to upload artifacts / models here:
https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts.py
https://github.com/allegroai/clearml/blob/master/examples/reporting/artifacts_retrieval.py
https://github.com/allegroai/clearml/blob/master/examples/reporting/model_reporting.py

you can specify the destination of the uploading like that :
when you initiate a task, you can set the parameter output_uri. If you set it to True, then the model will be uploaded to the uri specified in your conf file. Youcan also directly specify an url or you can use OutputModel.set_default_upload_uri or set_upload_destination ( https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#outputmodelset_default_upload_uri or https://clear.ml/docs/latest/docs/references/sdk/model_outputmodel#set_upload_destination )

Oh

David,
I haven't provided a monitor_model parameter

How do I provide an output storage destination for that stage of the pipeline?

But it seems to upload the model on noticing torch.save irrespective

Do you mean from within a pipeline ? Do you manually report the model ? It might point to a local file, especially if it has been auto logged. That is what happens when you are saving your model (thus to the local file system) from your script.

Can PipelineDecorator.upload_model be used to store models on the clearml fileserver?

Yep, that's exactly what's happening.
However here's what I want to do:
upload model to clearml’s fileserver and get the model url in the details for easy download

yes but it is supposed to be logged in the task corresponding to the step the model is being saved from. monitor_model makes the logging to the main pipeline task.

Also, does clearml by default upload models if we save them using torch.save?

So I am able to access it via sending requests to the clearml fileserver but, any way to access it from the dashboard(the main app)?

Hi WickedElephant66
When you are in the Projects section of the WebApp (second icon on the left), enter either "All Experiments" or any project you want to access to. Up on the center is the Models section. You csn find the url the model can be downloaded from, in the details, section

So I did exactly that, and the name and path of the model on the local repo is noted;
However, I want to upload it to the fileserver

So the issue is that the model url points to the file location on my machine,
Is there a way for me to pass the model url something else?

Hey, thanks for the reply
I have another question ;
Are Kwargs supported in functions decorated as a pipeline component?