Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone! I Discovered That Uploading Model Artifacts At Each Checkpoint To The Clearml Server Significantly Slows Down Training. So I Set

Hi everyone!
I discovered that uploading model artifacts at each checkpoint to the clearml server significantly slows down training. So I set output_uri to save everything on my local disk. However, I still want to upload the final model to clearml. Is there a way to do that after training? I found instructions for Pandas DataFrames, Local files, dictionaries, Folders, Numpy objects, and Image files, but not for models.

  
  
Posted 11 months ago
Votes Newest

Answers 6


@<1580005325879119872:profile|SweetCat82> , once an experiment finished running you can't change it's status unless you reset it. I think task.upload_artifact needs to come before your task finishes.

How did you try calling it? Fetching the task via SDK and then trying to upload the artifact?

  
  
Posted 11 months ago

Thank you for your help, @<1576381444509405184:profile|ManiacalLizard2> ! I tried it and got an error "Invalid task status: expected=created, status=completed ...".
I'm a bit confused about this... How to add an artifact to this task? Is it safe to call task.init() again?

  
  
Posted 11 months ago

@<1523701070390366208:profile|CostlyOstrich36> That's literally one of my first experiments ☺ . I trained a model in Jupyter and after the training stopped, ran 'task.upload_artifact' in the next cell.

  
  
Posted 11 months ago

you should be able to explicitly upload a file of your choice as artefact using something like this: None

  
  
Posted 11 months ago

you should know where your latest model is located then just call task.upload_artifact on that file ?

  
  
Posted 11 months ago

Will it be recognized as a model?
Though I will try it anyway, thank you!

  
  
Posted 11 months ago
586 Views
6 Answers
11 months ago
11 months ago
Tags