Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Guys, Is There A Ready Script That Can Delete All Models From S3 (Or Other Storage) That Are Related To Deleted Or Archived Experiments?

hey guys, is there a ready script that can delete all models from S3 (or other storage) that are related to deleted or archived experiments?

  
  
Posted 2 years ago
Votes Newest

Answers 11


we're using the latest version of clearml, clearml agent and clearml server, but we've been using trains/clearml for 2.5 years, so there are some old tasks left, I guess 😃

  
  
Posted 2 years ago

🙂

  
  
Posted 2 years ago

thanks!

  
  
Posted 2 years ago

two more questions about cleanup if you don't mind:
what if for some old tasks I get WARNING:root:Could not delete Task ID=a0908784a2a942c3812f947ec1f32c9f, 'Task' object has no attribute 'delete'? What's the best way of cleaning them? What is the recommended way of providing S3 credentials to cleanup task?

  
  
Posted 2 years ago

what if cleanup service is launched using ClearML-Agent Services container (part of the ClearML server)? adding clearml.conf to the home directory doesn't help

  
  
Posted 2 years ago

What is the recommended way of providing S3 credentials to cleanup task?

cleaml.conf or OS environment (AWS_ACCESS_KEY_ID ...)

  
  
Posted 2 years ago

oh wow, I didn't see delete_artifacts_and_models option

I guess we'll have to manually find old artifacts that are related to already deleted tasks

  
  
Posted 2 years ago

what if cleanup service is launched using ClearML-Agent Services container

The easiest is to use the container args and pass the AWS credentials as env variables:
-e AWS_ACCESS_KEY_ID=abcd -e ....Make sense ?

  
  
Posted 2 years ago

what if for some old tasks I get WARNING:root:Could not delete Task ID=a0908784a2a942c3812f947ec1f32c9f, 'Task' object has no attribute 'delete'? What's the best way of cleaning them?

This seems like an old SDK no?

  
  
Posted 2 years ago

we already have cleanup service set up and running, so we should be good from now on

  
  
Posted 2 years ago