Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
We Do Want To Have Control Which Files Are Logged In The Model Registry. There Is Such Option In Task.Init(), Auto_Connect_Frameworks=False Or Injecting A Dict() To It.. But Our Tasks Are Created As Being Part Of A Pipeline With Add_Function_Step(). So We

We do want to have control which files are logged in the model registry. There is such option in Task.init(), auto_connect_frameworks=False or injecting a dict() to it.. But our tasks are created as being part of a pipeline with add_function_step(). So we have no control for this parameter,auto_connect_frameworks, as we do not call Task.init() explicitly. Any workaround? Other ways we are flooded with files in our model registry.

  
  
Posted one year ago
Votes Newest

Answers 2


Hi @<1543766544847212544:profile|SorePelican79> ! You could use the following workaround:

from clearml import Task
from clearml.binding.frameworks import WeightsFileHandler
import torch


def filter_callback(
    callback_type: WeightsFileHandler.CallbackType,
    model_info: WeightsFileHandler.ModelInfo,
):
    print(model_info.__dict__)
    if (
        callback_type == WeightsFileHandler.CallbackType.save
        and "filter_out.pt" in model_info.local_model_path
    ):
        return None
    return model_info


if __name__ == "__main__":
    Task.init(project_name="example", task_name="filter out")
    WeightsFileHandler.add_pre_callback(filter_callback)
    filter_out_model = torch.nn.Module()
    dont_filter_out_model = torch.nn.Module()
    torch.save(filter_out_model, "filter_out.pt")
    torch.save(dont_filter_out_model, "dont_filter_out_model.pt")

As you can see, you can use WeightsFileHandler after the task has been initialized

  
  
Posted one year ago

Hi Eugen, very helpful, thank you!

  
  
Posted one year ago