I'M Using Docker To Run Experiments Which Should Have All Required Dependencies Necessary, But When Running On A Worker Node, The Clearml Pulls The Docker Image, Then Tries To Recreate The Environment. How Do You Override This Behavior? I Thought That By

Answered

I'm using docker to run experiments which should have all required dependencies necessary, but when running on a worker node, the ClearML pulls the docker image, then tries to recreate the environment. How do you override this behavior?

I thought that by setting agent.package_manager.system_site_packages = true and stopping and restarting the agent, this would happen, but the agent still tries to re-install everything in a virtualenv.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					ProudChicken98
				
					0
					 × 1

Votes Newest

Answers 3

Hi ProudChicken98 , although the agent will create a new venv, it will only install what's required - if the docker image comes preinstalled with your requirements, and you've set agent.package_manager.system_site_packages = true , you should only see the agent verifying these packages exist (i.e. it will list them), but not install them.

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

If you see a different behaviour, please share the logs :)

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Hi ProudChicken98 ,

You want the agent NOT to create a new env?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					TimelyPenguin76
				
					0
					 Administrator

Write your answer

561 Views

3 Answers

2 years ago

one year ago