Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi, When Using A Pipeline Decorator How Do I Add A Local Module? I Keep On Getting This Error Message:

When using a pipeline decorator how do i add a local module?
i keep on getting this error message:

Traceback (most recent call last):
  File "/tmp/tmpjwvz_hrj.py", line 35, in <module>
    results = train_model_task(**kwargs)
  File "/tmp/tmpjwvz_hrj.py", line 10, in train_model_task
    import train
ModuleNotFoundError: No module named 'train'

and this is my pipeline component:

@PipelineDecorator.component(return_values=['model_dir', 'epoch', 'task_id'], cache=False, task_type=TaskTypes.training)
def train_model_task(config_path, clearml_queue):
    print('Training model')
    import train

    model_dir, epoch, task_id = train.train_model(config_path=config_path, clearml_queue=clearml_queue)
    return model_dir, epoch, task_id
Posted 5 months ago
Votes Newest

Answers 7

Hi @<1523701295830011904:profile|CluelessFlamingo93> , I think you need this module as a part of repository, otherwise, how will the pipeline know what code to use?

Posted 5 months ago

@<1523701295830011904:profile|CluelessFlamingo93> , I'm not sure what you mean. Whenever you run a pipeline code (pipeline from decorators) if it's from a repository that repo will be logged. Where are you importing "train" from? What if you import entire package & point to the specific module?

Posted 5 months ago

@<1523701070390366208:profile|CostlyOstrich36> my repo is like this and both the files are located at the same dir so its weird that they cannot find train:

├── pytorch
├── tensorflow
│   ├── Project A
│   │   └── src
│   ├── Project B
│   │   ├── data
│   │   ├── model
│   │   ├── reports
│   │   └── utils
│   ├── hand_validator_boxes
│   │   ├── src
│   │   ├── train.py (the module i need)
│   │   └── clearml_pipeline.py (where the pipeline is initilizied
└── utils
Posted 5 months ago

@<1523701087100473344:profile|SuccessfulKoala55> and @<1523701070390366208:profile|CostlyOstrich36> , in the end I've found the problem, it was due to me running the pipeline locally and when running the pipeline locally it, doesn't copy all the dir but only the script that is running None

Posted 5 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36> , it is part of the repository, do pipelines run differently then normal tasks? what I mean is when i run a task it has a working directory do pipelines also have that or are their working directory the root of the repo?

Posted 5 months ago

@<1523701087100473344:profile|SuccessfulKoala55> yes the working dir is set to the correct path and yet it cannot import the train module

Posted 5 months ago

@<1523701295830011904:profile|CluelessFlamingo93> is the working dir set to the correct path? (Where the train.py file is)

Posted 5 months ago