Try to set CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true in the terminal start clearml-agent
See None
in my case, I set eveything inside the container, including the agent and not using docker mode altogether.
When my container start, it start the agent inside it in "normal" mode
ok, interesting overloading of that param, looking at the source code confirms that behavior
yeah, I was using the system python initially
depend on how the agent is launched ...
there is a tricky thing: clearml-agent should not be running from a venv itself ... don't remember where I read that doc
hmm, interesting, I was just sepcifying CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true
one specify the venv python, the other tell it to not do anything
ok, thx, need to think about this a bit more
I use CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/vemv/bin/python3.12 and it work for me
in that case yes. What happen is that in docker mode:
you run a clearml agent, that then receive a task
create a container
install another agent inside that container
then run that second agent inside the container
that second agent then pull the task and do the usuall build/install
CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL=true need to be set on that second agent somehow ...
inside the script that launch the agent, I set all the env need (aka disable installation with the var above)
Hmm, it seems like it's still trying to pip install my requirements int he container
but I couldn't get it to use my pre-created virutalenv
so i guess it need to be set inside the container
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/path/to/my/vemv/bin/python3.12 clearml-agent bla
do I need to set it in the container as well?
it always wanted to use the system python
guess I could set it in the dockerfile
So, removed docker to debug - I couldn't get the agent to use the python and virtualenv I wanted without also running the agent from that same virtualenv