Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I'M New Here, Hello Everyone

I'm new here, hello everyone 🙂
I have a question about pipelines. In my project, I try to run some code doing stuff, via a ClearML pipeline, in a ClearML agent/ docker mode.

clearml-agent daemon --docker registry/my_image

I need the docker image because do_stuff() needs specific things in the image as dependencies.
I run the clearml-agent on my own computer.
The code launching the pipeline looks like this:

# my_proj/pipeline.py

from my_module.stuff import do_stuff
from clearml import PipelineController


pipe = PipelineController(
    ...
)

pipe.add_function_step(function=do_stuff)

pipe.start()

When I run that code on my own computer, I can see the pipeline ongoing on ClearML server. The first step begins and fails with:

Traceback (most recent call last):
  File "/root/.clearml/venvs-builds/3.8/code/pipeline.py", line 5, in <module>
    from my_module import do_stuff
ModuleNotFoundError: No module named 'my_module'

Now I wonder how the clearml agent is supposed to find that code.
In my understanding, clearml agent creates a venv inside the docker container, installing clearml stuff and then run pipeline.py (which I don't know how it appears here). It doesn't seem to git clone anything.

I need your help to understand that.
Should the code be embedded in the docker image? Any alternative? (map volumes, ...) How to set path to code to execute ?

Thanks

  
  
Posted 4 months ago
Votes Newest

Answers 2


Hi @<1523701070390366208:profile|CostlyOstrich36> . Thanks.
Yes it is. With some slight modification, clearml agent finally tries to git clone the repo.
But it does that after some execution of part of my pipeline.py , so I think I get something wrong.

  
  
Posted 4 months ago

Hi @<1716263152817016832:profile|TartHare54> , is the module part of some repository?

  
  
Posted 4 months ago
421 Views
2 Answers
4 months ago
4 months ago
Tags
Similar posts