Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I have a problem: I’m upload my model weight on a self hosted clearml server, using the update_wights method. I was thinking that would delete the old weights from the file server once they get updated, since I can only downloaded the latest one from the

I have a problem: I’m upload my model weight on a self hosted clearml server, using the update_wights method. I was thinking that would delete the old weights from the file server once they get updated , since I can only downloaded the latest one from the webui, but they are still there, consuming space. Is this the expected behavior ? How can I get rid of those old files? I don’t think it has anything to do with the cleanup service, as it deletes the old tasks, but I want to delete the unnecessary files of a task that is still alive

  
  
Posted 2 months ago
Votes Newest

Answers 4


Yeah, but it doesn’t make sense to the file name of the file to be uploaded to have any sort of influence over the process, since the models have specifically a name to identify them. Moreover, if those files remain present on the server, but inaccessibile through the api, it’s a bug in any case, I’d say

  
  
Posted 2 months ago

was thinking that would delete the old weights from the file server once they get updated,

If you are uploading it to the same Task, make sure the model name and the filename is the same and it will override it (think filesystem filenames)

but they are still there, consuming space. Is this the expected behavior? How can I get rid of those old files?

you can programatically also remove (delete) models None

  
  
Posted 2 months ago

Hi @<1792727007507779584:profile|HollowKangaroo53> , if you only want to delete old checkpoints you would need to write some automation for that. I'm guessing that the cleanup example can be a good baseline. Then just fetch the artifacts list from tasks and use StorageManager to delete the relevant files from the files server.

  
  
Posted 2 months ago

I don’t thinks it’s possible : the checkpoints are output models, and such can be accessed from the task, but from the model class I can get only the url of the last version (as I expected from using update weights, as it does sound like it replaces the old weights with the new ones). Also, even if I could get the urls of the old checkpoints, how can I delete them from the file server? StorageManager doesn’t seem to have any method to delete remote files

  
  
Posted 2 months ago
317 Views
4 Answers
2 months ago
2 months ago
Tags
Similar posts