ClearML FAQ | Hi, How Does The Agent Determine Which Packages To Install? I Have

Answered

Hi, How Does The Agent Determine Which Packages To Install? I Have

Hi,
How does the agent determine which packages to install? I have google-cloud-bigquery in my requirements.txt, but when I run my pipeline on the agent (I install agent by helm on my k8s), it error on ModuleNotFoundError: No module named 'google' . I remove everything except google-cloud-bigquery in my requirements.txt, but the list of INSTALLED PACKAGES does not change.
Thanks

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					PunyWoodpecker71
				
					0
					 × 1

Votes Newest

Answers 5

The setting is in the agent clearml.conf file

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

By default, the agent will use the packages detected when you ran your code locally (which is stored in the installed packages section). You can force the agent to use only the requirements.txt file from the repository

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

and I installed the google package locally, so it's weird that it didnt detect it

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					GrittyCormorant73
				
					0
					 × 1

If you installed it and did not import it (or did not use it even though you imported it) it will probably not be detected

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

SuccessfulKoala55 Thanks Jake. Is t hat an option on the helm values or I set in the pipeline decorator?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					GrittyCormorant73
				
					0
					 × 1

Write your answer

1K Views

5 Answers

2 years ago

one year ago

Tags

Similar posts

What The Best Practice To Configure Clearml-Agent In The Way To Don’T Install Packages That Are Already Available Every Time? I Use Nvidia-Docker To Run The Train Script For Example. I Initialized Clearml-Agent Inside A Running Nvidia-Docker Container

Hi! When Running A Remote Task On The Agent, Clearml Installs Additional System Packages. How Can This Be Disabled? The Variable Agent.Package_Manager.System_Site_Packages Does Not Work

Hello!, Does Anyone Know Why When Running A Task In A Specific Docker Image, There Are Some Python Packages Whose Versions Doesn'T Match? For Example, I Have Installed The Package Ultralytics On 8.1.25 On The Docker Image But It'S On 8.1.11 In The Install

Hi, I’M Running An Agent In K8S Using The Glue. I’M Trying To Set The Setup Shell Script To Install Some Packages Through Apt But I Don’T Think It Even Runs. I Dont See Anything In The Console Logs That Says It Does, At Least. Is There Anyway I Can Debug

Clearml Agent Can Work On Gpu Machine In No Internet Environment Where We Have Proxy For Pip Packages And Ubuntu Updates ? If Yes, How To Configure These Proxy In Agent Settings. I See At Launch Of Script Saying "Hello" It Install Many Packages Befor

Hello Everyone - I Have Installed A New Clearml Server, Now I Want To Run My Scripts Using The 4 Gpus On That Server Machine. Do I Need To Install A Clearml Agent On That Machine, Or Does The Clearml Server Come With A Built-In Agent Automatically?

After Resetting And Enqueueing A Task, The Pip Installation Procedure In A Remote Agent Tries To Install Packages Different From Packages That Were Installed In The Execution Session Before The Resetting And Enquiring Of The Task. How Can I Control That

Is There A Way To Use Clearml-Agent With An Existing Condav Venv? I Don'T Want It To Create A New Venv Or Install Any Packages.

Hi, I Want Clearml To Not Install Packages From My Pycharm Environment, But Activate An Environment I Have Locally On The Computer The Agent Is Installed On, Is The Variable Clearml_Agent_Skip_Pip_Venv_Install Relevant To This, And If So Can I Configure I

Hi, I'M Trying To Run A Pipeline Object Using Clearml Agent On Mac Os And It Fails. I See In The Error I Get That It Doesn'T Succeed To Install The Relevant Packages. When I Run Each Script Separately, It Works Well; And When I Run The Pipeline In Linux,