docstring ?
Usually the preferred way is StorageManager
https://clear.ml/docs/latest/docs/references/sdk/storage
https://clear.ml/docs/latest/docs/integrations/storage
AgitatedDove14 thank you. Maybe you know about OutputModel.remove
method or something like that?
StorageManager
class seems to lack methods for removal as well
AgitatedDove14 are models technically Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
StorageManager
Oh it has no remove 😞StorageHelper.delete
is the only way
are models technically
Task
s and can they be treated as such? If not, how to delete a model permanently (both from the server and from AWS storage)?
When you call Task.delete() it actually goes over a;; the models/artifacts and deletes them from the storage
AgitatedDove14 by task you mean the training task or the separate task corresponding to the model itself? The former won't work since I don't want to delete the training task, only the models
I just happened to spawn multiple OutputModels
within a single script which is being run in a single task. That is, I see dozens of models in Models
tab in web UI. What I want is to delete most of them (along with the files in S3), preserving the spawning task
Hi MelancholyElk85
I think you are right, OutputModel is missing, remove
method.
Maybe we should have a class method on Model , something like:@classmethod Model.remove(model: Union[str, Model], delete_weights_file: bool, force: bool): # actually remove model and weights file
wdyt?
AND also add task_name
parameter to Model.query_models
As for now it seems to query only by project name and model name.
so that the way of doing it would be like this:all_models = Model.query_models(projeect_name=..., task_name=..., tags=['running-best-checkpoint']) all_models = sorted(all_models, key=lambda x: extract_epoch(x)) for model in all_models[:-num_to_preserve]: Model.remove(model, delete_weights_file=True)
MelancholyElk85 that looks great, let me see how quickly we can push it (I think 1.1.5 needs to be pushed very soon, I'll check if we can have it before 🙂 )