Hey Everyone, What Is The Best Way To Export Environment Variables (Especially The Ones With Secrets Such As Db Credentials) And Make Them Available To A Clearml Agent In The Cloud? I Want To Create A Task Which Will Essentially Use A Set Of Db Credenti

Answered

Hey everyone,

What is the best way to export environment variables (especially the ones with secrets such as DB credentials) and make them available to a clearml agent in the cloud?

I want to create a task which will essentially use a set of DB credentials to query the PostgreSQL database and create CSV file which will later be used for another task for training purposes.

Any help on this topic is appreciated!

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					TestyKitten53
				
					0
					 × 1

Votes Newest

Answers 4

For us, we use Azure, we use KeyVault to store secret.
The VM/node that run agent have a Azure Identity that have permission to read those Secret.
To pull the Secret, we simply have az login --identity [--client-id foobar] prior to az secret ....

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					ManiacalLizard2
				
					0
					 × 1

Hmm .. Understood .. I could use GCP in my case

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					TestyKitten53
				
					0
					 × 1

pretty sure GCP have all the equivalent

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					ManiacalLizard2
				
					0
					 × 1

For local agent running on-prem, we use Service Principal or each user login to auth with Azure and then mount ~/.azure into the container

  				
Posted 
	3 months ago

					More
				  		
  Report
		
					ManiacalLizard2
				
					0
					 × 1

Write your answer

573 Views

4 Answers

3 months ago