Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?

Yes, I have tested it through another toy example and it reused the first run.

I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.

Hi TimelyMouse69 I am not going to cache the virtual env but reuse a preinstalled one. I have set agent.python_binary to use the virtual env python, but clearml use it to create another new virtual env not the preinstalled one. I have install the virtual env before I use clearml, so I want to reuse it.

What I want is all the experiments use the same codes and the preinstalled python virtual env.

After I enqueue the experiment task into a queue, clearml agent always creates a new venv and installs pakcages.

And in the logs?

Hi AgitatedDove14 can you give me some help?

Of course I did.

So could you just setup your virtual environment with a task?

Hi TimelyMouse69 can you give any advice? Or can somebody else help? Thanks in advance.

The logs outputs the 10 epoch's evaluation result and stops there. But it should run 50 epochs.

I set the environment variable export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/general-user/frank/mlflow_ray/env/bin/python3 , then I clone a task and run it. This time no more reinstallation, the task use /general-user/frank/mlflow_ray/env/bin/python3 to run the experiment. But it stuck in the epoch10 when the first evaluation over and just stops there.

Even I only set agent.venvs_cache.path=/home/frank/env in the clearml.conf, the train still failed to start. ClearML always create a new venvs and install Cython and some other packages, so weird!
... Installing collected packages: Cython Successfully installed Cython-0.29.32 Adding venv into cache: /general-user/frank/.clearml/venvs-builds/3.8 Running task id [42a050853c43445ebf9248bd4aa54091]: [.]$ /general-user/frank/.clearml/venvs-builds/3.8/bin/python -u tools/train.pyI am confused totally!!!

To use a specific binary you can set in in the config: https://clear.ml/docs/latest/docs/configs/clearml_conf/#:~:text=python%20version%20(default)-,agent.python_binary,-(string)

But if you're trying to cache virtual environments you might be more interested in: https://clear.ml/docs/latest/docs/clearml_agent#environment-caching

ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
https://clear.ml/docs/latest/docs/clearml_agent#virtual-environment-reuse

I have install a python environment by virtualenv tool, let's say

/home/frank/env

and python is

/home/frank/env/bin/python3.

How to reuse the virtualenv by setting clearml agent?

So the agent is already caching the entire venv for you, nothing to worry about, just make sure you have this line in clearml:
https://github.com/allegroai/clearml-agent/blob/249b51a31bee97d63f41c6d5542e657962008b68/docs/clearml.conf#L131
No need to provide it an existing venv, if you are running in docker mode, you can provide
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/home/frank/env/bin/python3

Not in docker mode. So I just need to set venvs_cache.path=/home/frank/env ? I do not think this works.