MelancholyElk85 that looks great, let me see how quickly we can push it (I think 1.1.5 needs to be pushed very soon, I'll check if we can have it before 🙂 )
so that the way of doing it would be like this:all_models = Model.query_models(projeect_name=..., task_name=..., tags=['running-best-checkpoint']) all_models = sorted(all_models, key=lambda x: extract_epoch(x)) for model in all_models[:-num_to_preserve]: Model.remove(model, delete_weights_file=True)
As for now it seems to query only by project name and model name.
AND also add task_name
parameter to Model.query_models
Hi MelancholyElk85
I think you are right, OutputModel is missing, remove
method.
Maybe we should have a class method on Model , something like:@classmethod Model.remove(model: Union[str, Model], delete_weights_file: bool, force: bool): # actually remove model and weights file
wdyt?
I just happened to spawn multiple OutputModels
within a single script which is being run in a single task. That is, I see dozens of models in Models
tab in web UI. What I want is to delete most of them (along with the files in S3), preserving the spawning task
AgitatedDove14 by task you mean the training task or the separate task corresponding to the model itself? The former won't work since I don't want to delete the training task, only the models
are models technically
Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
When you call Task.delete() it actually goes over a;; the models/artifacts and deletes them from the storage
StorageManager
Oh it has no remove 😞StorageHelper.delete
is the only way
AgitatedDove14 are models technically Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
StorageManager
class seems to lack methods for removal as well
AgitatedDove14 thank you. Maybe you know about OutputModel.remove
method or something like that?
docstring ?
Usually the preferred way is StorageManager
https://clear.ml/docs/latest/docs/references/sdk/storage
https://clear.ml/docs/latest/docs/integrations/storage