I also tried to build my own image and uploaded it to harbor, but the tasks didn't use the packages from it.
FROM nvidia/cuda:12.4.0-devel-ubuntu22.04

ENV TZ=Europe/Moscow

RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

WORKDIR /workspace

RUN apt-get update && apt-get install -y
software-properties-common
curl &&
add-apt-repository ppa:deadsnakes/ppa -y &&
apt-get update && apt-get install -y
python3.11
python3.11-venv
python3.11-dev &&
rm -rf /var/lib/apt/lists/*

RUN curl -sS None | python3.11

RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.11 1

RUN python3.11 -m venv /opt/venv
ENV VIRTUAL_ENV=/opt/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

COPY . .

RUN pip install --upgrade pip
RUN pip install torch --index-url None
RUN pip install clearml --index-url None

RUN pip install -r requirements.txt --index-url None

yes thanks , but if I do this, the packages will be installed for each step again, is it possible to use a single venv?

Notice that the venv is Cached on the clearml-agent host machine (if this is k8s glue, make sure to setup the Cache as a PV to achieve the same)
This means there is no need to worry about that and this is stable.
That said, if you have an existing VENV inside the container, just add docker_args="-e CLEARML_AGENT_SKIP_PIP_VENV_INSTALL =/path/to/bin/python"
See here: None

BTW: notice that in your example you are probably installing to the system path not the venv path, and to your point specifically, add ENV=CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=$VIRTUAL_ENV

Lastly if you want your agent to Alwoays use the activated Venv inside the container set:

docker_use_activated_venv=true

None

Hi ImmenseMole52

but tasks of this pipeline dont inherit docker and packages, why? I want to build or pull one docker and env for all pipeline steps only once, so ow can i do it?

you have to specify the docker image for the pipeline Tasks, by default it will not assume it is the same as the pipeline controller, basically just pass:

pipe.add_function_step(
        name="load_data",
        function=load_data,
        function_kwargs={"config": config["data"]},
        docker="MY CONTAINER HERE"
    )

make sense?

Thanks!

AgitatedDove14 yes thanks , but if I do this, the packages will be installed for each step again, is it possible to use a single venv? I use pytorch/pytorch:2.6.0-cuda12.4-cudnn9-runtime and install some extra packages