It is. In what format should I specify it? Would this enforce that package on various components? Would it then no longer capture import statements?

There's code that strips the type hints from the component function, just think it should be applied to the helper functions too :)

We can change the project name’s of course, if there’s a suggestion/guide that will make them see past the namespace…

We have an internal mono-repo and some of the packages are required - they’re all available correctly for the controller, only some are required for the individual tasks, but the “magic” doesn’t happen 😞
That is, the controller does not identify them as a requirement, so they’re not installed in the tasks environment.

I think this is the main issue, is this reproducible ? How can we test that?

Yes. Though again, just highlighting the naming of

foo-mod

is arbitrary. The actual module simply has a folder structured with an implicit namespace:

Yep I think this is exactly why it fails detecting it, let me check that

And it’s failing on typing hints for functions passed in

pipe.add_function_step(…, helper_function=[…])

… I guess those aren’t being removed like the wrapped function step?

Can you provide the log? I think I'm missing what exactly was added into the decorator that somehow fails the Task creation

PricklyRaven28 That would be my fallback, it would make development much slower (having to build containers with every small change)

The only thing I could think of is that the output of pip freeze would be a URL?

Using the PipelineController with add_function_step

We also wanted this, we preferred to create a docker image with all we need, and let the pipeline steps use that docker image

That way you don’t rely on clearml capturing the local env, and you can control what exists in the env

is this repo installed on the machine creating the pipeline ?
You can also manually add it here `packages={"link_to_internal_python_package",]
None

How or why is this the issue?

The main issue is a missing requirement on the Task component, and this is why it is failing.
You can however manually specify package (and I'm assuming this will solve the issue), but it should have autodetected, no?