Hmmmmm no that's a bad solution, you're right, maybe SuccessfulKoala55 , might have an idea?

VividDucks43 , I think I might have misunderstood you a bit - For a single pipeline you would have the same requirements.txt, why would you need many?

So the idea of using different set of packages for pipelines in clearml is to have many different requirements.txt files?

I was also curious if I've missed an easy way to mention required packages for the controller, like you can do that in components (by providing

packages=[...]

).

I think you can do this using https://clear.ml/docs/latest/docs/references/sdk/task#taskadd_requirements
I'm not sure how the pipeline controller is handled but it still should be regarded as a task on a deeper level.

Meaning that you might need to use task = Task.current_task() and then task.add_requirements(package_name="your_package_name")
https://clear.ml/docs/latest/docs/references/sdk/task#taskcurrent_task

I'm just thinking If I have multiple pipelines with different purposes controllers don't have the same package requirements (some of them may take a lot of time) therefore you would need multiple files for each one. Maybe the idea of having multiple pipelines for one project is wrong - I don't actually know 🤔

I was also curious if I've missed an easy way to mention required packages for the controller, like you can do that in components (by providing packages=[...] ).

Hi VividDucks43 ,

I think what you're looking for is this:
https://clear.ml/docs/latest/docs/references/sdk/task#taskforce_requirements_env_freeze
🙂