Hi Guys, I Encountered A Very Frustrating Problem About Virtual Enviroment Management When I Use Clearml. I Use Virtualenv To Create A New Python Env And Installed All The Needed Python Pakcages In The Env. The Env Path Is

Answered

Hi guys, I encountered a very frustrating problem about virtual enviroment management when I use ClearML. I use virtualenv to create a new python env and installed all the needed python pakcages in the env. The env path is /general-user/frank/mlflow_ray/env (so the python binary is /general-user/frank/mlflow_ray/env/bin/python ). I source the env and run python trian.py ... and the task and all the metrics and models are logged into ClearML webapp.

Now I clone the experiment/task on the ClearML webapp, and clear all the preinstall packages, and enqueue the task into a queue. The task is executed but failed to run because ClearML want to create a another new virtual env and it do not reuse I preinstalled one.

So, how can I set ClearML conf to reuse the preinstalled one?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ThoughtfulBadger56
				
					0
					 × 1

Votes Newest

Answers 7

Yes, I think you are right. Just set export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL= /home/bar/env/bin/python3 .

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ThoughtfulBadger56
				
					0
					 × 1

I don't think you need to mix. For example if you have a pre-prepared environment then it should something like export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=<PATH_TO_ENV_BINARY>

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Thank you CostlyOstrich36 What's the difference between the two environment variables?
Shall I set export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/home/bar/env/bin/python3 or
export CLEARML_AGENT_SKIP_PYTHON_VENV_INSTALL=/home/bar/env/bin/python3 or
export CLEARML_AGENT_SKIP_PIP_VENV_INSTALL=/home/bar/env and
export CLEARML_AGENT_SKIP_PYTHON_VENV_INSTALL=/home/bar/env/bin/python3

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ThoughtfulBadger56
				
					0
					 × 1

None of them works... CostlyOstrich36 😵‍💫 😵‍💫 😵‍💫

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ThoughtfulBadger56
				
					0
					 × 1

Hi ThoughtfulBadger56 ,

I think these are a the environment variables you're looking for:
CLEARML_AGENT_SKIP_PIP_VENV_INSTALL CLEARML_AGENT_SKIP_PYTHON_ENV_INSTALL
https://clear.ml/docs/latest/docs/clearml_agent/clearml_agent_env_var

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

I think the 3rd one, let me know what worked for you

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

The official doc just gives too much choices but less details about how to set them and the relationships.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ThoughtfulBadger56
				
					0
					 × 1

Write your answer

2K Views

7 Answers

2 years ago