From the environment variable

Oh, so you mean CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/venv/bin/python3 ??

So I tried:

CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/data/hieu/opt/python-venv/fastai/bin/python3.10
clearml-agent  daemon  --queue no_venv

Then enqueue a cloned task to no_venv
It is still trying to create a venv (and fail):

[...]
tag = 
docker_cmd = 
entry_point = debug.py
working_dir = apple_ic
created virtual environment CPython3.10.10.final.0-64 in 140ms
  creator CPython3Posix(dest=/data/hieu/deleteme/clearml-agent/venvs-builds/3.10, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mhtrinh/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Using cached repository in "/home/mhtrinh/.clearml/vcs-cache/KitchenSink.44ea8add86cace4fd712971ff93b965c/KitchenSink"
[...]
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-22.2.2
Collecting Cython
  Using cached Cython-3.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
Installing collected packages: Cython
Successfully installed Cython-3.0.0
Adding venv into cache: /home/mhtrinh/.clearml/venvs-builds/3.10
Running task id [4a413e955b6e47c38939e0b1fe5a8786]:
[apple_ic]$ /home/mhtrinh/.clearml/venvs-builds/3.10/bin/python -u debug.py
Summary - installed python packages:
pip:
- Cython==3.0.0
Environment setup completed successfully
Starting Task Execution:
Traceback (most recent call last):
  File "/data/hieu/deleteme/clearml-agent/venvs-builds/3.10/task_repository/KitchenSink/apple_ic/debug.py", line 2, in <module>
    import livsdk.livbatch
ModuleNotFoundError: No module named 'livsdk'
2023-08-14 20:38:06
Process failed, exit code 1

Yes

Found a trick to have empty Installed package:
clearml.Task.force_requirements_env_freeze(force=True,requirements_file="/dev/null")
Not sure if this is the right way or not ...

@<1523701070390366208:profile|CostlyOstrich36> Is there a way to tell clearml to not try to detect the Installed package ?

Hi @<1576381444509405184:profile|ManiacalLizard2> , I think this is the env var you're looking for
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
None

Should I put that in the clearml.conf file?

you should set it on the machine running the agent

no, it's an environment variable

But then how did the agent know where is the venv that it needs to use?

Set that env var in the terminal before running the agent ?

Yes

So I now just need to find a way to not populate Installed Package in the first place

oh, looks like I need to empty the Installed Package before enqueue the cloned task