Hi! We'Re Trying To Orchestrate An Experiment Through The Clearml Ui With A Remote Agent On K8S. Our Training Job Requires A Package Only Available On Our Internal Pypi Server. How Should We Go About This Because The Job Keeps Failing With:

Answered

Hi!

We're trying to orchestrate an experiment through the ClearML ui with a remote agent on k8s. Our training job requires a package only available on our internal pypi server. How should we go about this because the job keeps failing with:
ERROR: Could not find a version that satisfies the requirement our_internal_package==0.1.2rc3 (from -r /tmp/cached-reqsdipdv1ju.txt (line 6)) (from versions: none) ERROR: No matching distribution found for our_internal_package==0.1.2rc3 (from -r /tmp/cached-reqsdipdv1ju.txt (line 6)) clearml_agent: ERROR: Could not install task requirements!
I set the extra pypi server on the agent with the CLEARML_AGENT_PACKAGE_MANAGER_EXTRA_INDEX_URL env var, but it does not seem to be looking at it 🤔

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyFox64
				
					0
					 × 1

Votes Newest

Answers 4

Hi CostlyFox64 ,

Can you try configuring your ~/clearml.conf with the following?
agent.package_manager.extra_index_url= [ "https://<USER>:<PASSWORD>@packages.<HOSTNAME>/<REPO_PATH>" ]

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

Hi CostlyOstrich36 . Would it also be possible to set those values through env vars?

Because I am using the https://github.com/allegroai/clearml-helm-charts/blob/06070a5c20691aaf83fc919b1bf07a822c212d5a/charts/clearml/values.yaml#L330 on Kubernetes and can thus far only configure it through env variables

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyFox64
				
					0
					 × 1

Hi CostlyFox64 ,

I set the extra pypi server on the agent with the

CLEARML_AGENT_PACKAGE_MANAGER_EXTRA_INDEX_URL

env var, but it does not seem to be looking at it

The correct format for the env var name would be CLEARML_AGENT__AGENT__PACKAGE_MANAGER__EXTRA_INDEX_URL (note that the . is replaced by double-underline chars)

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Worked like a charm, thanks SuccessfulKoala55 !!! 😄

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					CostlyFox64
				
					0
					 × 1

Write your answer

1K Views

4 Answers

2 years ago

one year ago