Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Is There A Way To Tell The Agent To Use A Specific Venv Pre Installed ? Like The One Already Installed In The Developer Pc And The Agent Is Running Inside That Same Pc?

Is there a way to tell the agent to use a specific venv pre installed ? Like the one already installed in the developer PC and the agent is running inside that same PC?

  
  
Posted 9 months ago
Votes Newest

Answers 15


Hi @<1576381444509405184:profile|ManiacalLizard2> , I think this is the env var you're looking for
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL
None

  
  
Posted 9 months ago

Set that env var in the terminal before running the agent ?

  
  
Posted 9 months ago

Should I put that in the clearml.conf file?

  
  
Posted 9 months ago

But then how did the agent know where is the venv that it needs to use?

  
  
Posted 9 months ago

From the environment variable

  
  
Posted 9 months ago

So I tried:

CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/data/hieu/opt/python-venv/fastai/bin/python3.10
clearml-agent  daemon  --queue no_venv

Then enqueue a cloned task to no_venv
It is still trying to create a venv (and fail):

[...]
tag = 
docker_cmd = 
entry_point = debug.py
working_dir = apple_ic
created virtual environment CPython3.10.10.final.0-64 in 140ms
  creator CPython3Posix(dest=/data/hieu/deleteme/clearml-agent/venvs-builds/3.10, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mhtrinh/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.40.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Using cached repository in "/home/mhtrinh/.clearml/vcs-cache/KitchenSink.44ea8add86cace4fd712971ff93b965c/KitchenSink"
[...]
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-22.2.2
Collecting Cython
  Using cached Cython-3.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)
Installing collected packages: Cython
Successfully installed Cython-3.0.0
Adding venv into cache: /home/mhtrinh/.clearml/venvs-builds/3.10
Running task id [4a413e955b6e47c38939e0b1fe5a8786]:
[apple_ic]$ /home/mhtrinh/.clearml/venvs-builds/3.10/bin/python -u debug.py
Summary - installed python packages:
pip:
- Cython==3.0.0
Environment setup completed successfully
Starting Task Execution:
Traceback (most recent call last):
  File "/data/hieu/deleteme/clearml-agent/venvs-builds/3.10/task_repository/KitchenSink/apple_ic/debug.py", line 2, in <module>
    import livsdk.livbatch
ModuleNotFoundError: No module named 'livsdk'
2023-08-14 20:38:06
Process failed, exit code 1
  
  
Posted 9 months ago

Found a trick to have empty Installed package:
clearml.Task.force_requirements_env_freeze(force=True,requirements_file="/dev/null")
Not sure if this is the right way or not ...

  
  
Posted 9 months ago

Oh, so you mean CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/venv/bin/python3 ??

  
  
Posted 9 months ago

you should set it on the machine running the agent

  
  
Posted 9 months ago

oh, looks like I need to empty the Installed Package before enqueue the cloned task

  
  
Posted 9 months ago

So I now just need to find a way to not populate Installed Package in the first place

  
  
Posted 9 months ago

Yes

  
  
Posted 9 months ago

no, it's an environment variable

  
  
Posted 9 months ago

@<1523701070390366208:profile|CostlyOstrich36> Is there a way to tell clearml to not try to detect the Installed package ?

  
  
Posted 9 months ago

Yes

  
  
Posted 9 months ago