AgitatedDove14 are models technically Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
MelancholyElk85 that looks great, let me see how quickly we can push it (I think 1.1.5 needs to be pushed very soon, I'll check if we can have it before 🙂 )
I just happened to spawn multiple OutputModels
within a single script which is being run in a single task. That is, I see dozens of models in Models
tab in web UI. What I want is to delete most of them (along with the files in S3), preserving the spawning task
StorageManager
class seems to lack methods for removal as well
Hi MelancholyElk85
I think you are right, OutputModel is missing, remove
method.
Maybe we should have a class method on Model , something like:@classmethod Model.remove(model: Union[str, Model], delete_weights_file: bool, force: bool): # actually remove model and weights file
wdyt?
StorageManager
Oh it has no remove 😞StorageHelper.delete
is the only way
As for now it seems to query only by project name and model name.
AgitatedDove14 by task you mean the training task or the separate task corresponding to the model itself? The former won't work since I don't want to delete the training task, only the models
docstring ?
Usually the preferred way is StorageManager
https://clear.ml/docs/latest/docs/references/sdk/storage
https://clear.ml/docs/latest/docs/integrations/storage
are models technically
Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
When you call Task.delete() it actually goes over a;; the models/artifacts and deletes them from the storage
AND also add task_name
parameter to Model.query_models
AgitatedDove14 thank you. Maybe you know about OutputModel.remove
method or something like that?
so that the way of doing it would be like this:all_models = Model.query_models(projeect_name=..., task_name=..., tags=['running-best-checkpoint']) all_models = sorted(all_models, key=lambda x: extract_epoch(x)) for model in all_models[:-num_to_preserve]: Model.remove(model, delete_weights_file=True)