Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Is There A Way To Save The Models Completely On The Clearml Server? It Seems That Clearml Server Does Not Store The Models Or Artifacts Itself, But They Are Stored Somewhere Else (E.G., Aws S3-Bucket) Or On My Local Machine And Clearml Server Is Only Sto

Is there a way to save the models completely on the ClearML server?

It seems that ClearML Server does not store the models or artifacts itself, but they are stored somewhere else (e.g., AWS S3-bucket) or on my local machine and ClearML Server is only storing configuration parameters and previews (e.g., when the artifact is a pandas dataframe). Is that right?

  
  
Posted 2 years ago
Votes Newest

Answers 45


Exactly 🙂

  
  
Posted 2 years ago

@<1523701083040387072:profile|UnevenDolphin73> : If I do, what should I configure how?

  
  
Posted 2 years ago

We're certainly working hard on improving the documentation (and I do apologize for the frustrating experience)

  
  
Posted 2 years ago

missing a configuration option

Which one, where? Any idea? I did not set output_uri - do I have to do that?

I am refering to

  
  
Posted 2 years ago

I have already been trying to contribute (have three pull requests), but honestly I feel it is a bit weird, that I need to update a documentation about something I do not understand, while I actually try to evaluate if ClearML is the right tool for our company...

  
  
Posted 2 years ago

Hi @<1523704157695905792:profile|VivaciousBadger56> , you can configure Task.init(..., output_uri=True) and this will save the models to the clearml file server

  
  
Posted 2 years ago

The documentation is messy, I’ve complained about it the in the past too 🙈

  
  
Posted 2 years ago

@<1523701087100473344:profile|SuccessfulKoala55> Also, I think that - in this case, but also in other cases - the issue is not just the documentation, but also the design of the SDK.

  
  
Posted 2 years ago

From the one you sent - None

  
  
Posted 2 years ago

@<1523704157695905792:profile|VivaciousBadger56> I'm not sure I'm following you - is the issue not being able to upload to the ClearML server or to load the downloaded file?

  
  
Posted 2 years ago

I am not sure if it the fact the name of the file ends with .model is an issue - but that would be somewhat crazy design...

  
  
Posted 2 years ago

@<1523701083040387072:profile|UnevenDolphin73> : From which URL is your most recent screenshot?

  
  
Posted 2 years ago

We'll try to add referenced to that in other places as well 👍

  
  
Posted 2 years ago

@<1523701070390366208:profile|CostlyOstrich36>

My training outputs a model as a zip file. The way I save and load the zip file to make up my model is custom made (no library is directly used), because we invented the entire modelling ourselves. What I did so far:

output_model = OutputModel(task=..., config_dict={...}, name=f"...")
output_model.update_weights("C:\io__path\...", is_package=True)

and I am trying to load the model in a different Python process with

mymodel = task.models['output'][0]
mymodel = mymodel.get_local_copy(extract_archive=True, raise_on_error=True)

and I get in the clearml cache a . training.pt file, which seems to be some kind of archive. Inside I have two files named data.pkl and version and a folder with the two files named 86922176 and 86934640 .

I am not sure how to proceed after trying to use pickle, zip and joblib. I am kind of at a loss. I suspect, my original zip file might be somehow inside, but I am not sure.

Sure, we could simply use the generic artifacts sdk, but I would like to use the available terminological methods and functions.

How should I proceed?

  
  
Posted 2 years ago

Hi all, sorry for not being so responsive today 🙏

  
  
Posted 2 years ago
170K Views
45 Answers
2 years ago
2 years ago
Tags