from clearml import PipelineController

or how can I add some libraries I'd like it to pip install in the new environment

Have you run experiments with the SDK? i.e added Task.init()

I have a simple pipline that works

now the question how to make the production stuff to work

the only way any computer could figure this out is by running it

we have a diff name for this file in the repo

nice! Thank you

the question how does ClearML know to create env and what files does it copy to the task
/homes/yossi/.clearml/venvs-builds/3.7/

if a==True: import torch else: import tensorflow

based on what reqeuirment.txt manifest?

how this is possible?

locally and remotely

the question how does ClearML know to create env and what files does it copy to the task

Either automatically detecting the packages in requirements.txt OR using the packages listed in the task itself

The one sitting in the repository

PIP Mode

By default, ClearML Agent works in PIP Mode, in which it uses

as the package manager. When ClearML runs, it will create a virtual environment (or reuse an existing one, see

). Task dependencies (Python packages) will be installed in the virtual environment.

how does it know what are the dependencies of a task?

if this was possible we wouldn't need pip

I'm sorry. I think I wrote something wrong. I'll elaborate:
The SDK detects all the packages that are used during the run - The Agent will install a venv with those packages.
I think there is also an option to specify a requirements file directly in the agent.

Is there a reason you want to install packages from a requirements file instead of just using the automatic detection + agent?

I think I am missing something

I think this is what you're looking for - https://clear.ml/docs/latest/docs/references/sdk/task#taskforce_requirements_env_freeze

Task.add_requirements

Yep Correct

Hi AverageRabbit65 , can you elaborate on what you're trying to do?
ClearML-Agent will automatically create a venv and install everything

and just making sure - by pipeline we're talking about the ClearML pipelines, correct?
https://clear.ml/docs/latest/docs/references/sdk/automation_controller_pipelinecontroller

consider this case:

so it looks for requirement.txt?

And the pipeline runs with agents or locally?

it's an undecidable problem