To use a specific binary you can set in in the config: https://clear.ml/docs/latest/docs/configs/clearml_conf/#:~:text=python%20version%20(default)-,agent.python_binary,-(string)
But if you're trying to cache virtual environments you might be more interested in: https://clear.ml/docs/latest/docs/clearml_agent#environment-caching
ThoughtfulBadger56 Have you uncommented the existing venvs_cache section in the config file?
I have install a python environment by virtualenv tool, let's say
and python is
How to reuse the virtualenv by setting clearml agent?
So the agent is already caching the entire venv for you, nothing to worry about, just make sure you have this line in clearml:
No need to provide it an existing venv, if you are running in docker mode, you can provide
After I enqueue the experiment task into a queue, clearml agent always creates a new venv and installs pakcages.
Even I only set
agent.venvs_cache.path=/home/frank/env in the clearml.conf, the train still failed to start. ClearML always create a new venvs and install Cython and some other packages, so weird!
... Installing collected packages: Cython Successfully installed Cython-0.29.32 Adding venv into cache: /general-user/frank/.clearml/venvs-builds/3.8 Running task id [42a050853c43445ebf9248bd4aa54091]: [.]$ /general-user/frank/.clearml/venvs-builds/3.8/bin/python -u tools/train.pyI am confused totally!!!
Hi TimelyMouse69 I am not going to cache the virtual env but reuse a preinstalled one. I have set agent.python_binary to use the virtual env python, but clearml use it to create another new virtual env not the preinstalled one. I have install the virtual env before I use clearml, so I want to reuse it.
Not in docker mode. So I just need to set
venvs_cache.path=/home/frank/env ? I do not think this works.
Hi TimelyMouse69 can you give any advice? Or can somebody else help? Thanks in advance.
Hi AgitatedDove14 can you give me some help?
I set the environment variable
export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/general-user/frank/mlflow_ray/env/bin/python3 , then I clone a task and run it. This time no more reinstallation, the task use
to run the experiment. But it stuck in the epoch10 when the first evaluation over and just stops there.
Could you test the following:
Without reusing the virtual environment you made manually:
Can you run a task twice and see if the second run is at least reusing the virtual environment of the first run?
The logs outputs the 10 epoch's evaluation result and stops there. But it should run 50 epochs.
Yes, I have tested it through another toy example and it reused the first run.
So could you just setup your virtual environment with a task?
And in the logs?
Of course I did.
I'm afraid what you're trying to do isn't a supported implementation.
You'll have to choose between using docker mode to have one virtual environment for everything or using the pip mode where you can used the cached virtual environments but you can't reuse the one you currently have.
What I want is all the experiments use the same codes and the preinstalled python virtual env.