ClearML FAQ | Hi All, I Have A Quick Question Regarding Passing Secrets To A Docker Container. Our Set Up Is The Following: We Use Gcp Autoscaler To Spawn "Workers" Where The Training/Testing Is Done. In Order To Pass Secrets To The Docker Container Where The Code Is B

Answered

Hi All, I Have A Quick Question Regarding Passing Secrets To A Docker Container. Our Set Up Is The Following: We Use Gcp Autoscaler To Spawn "Workers" Where The Training/Testing Is Done. In Order To Pass Secrets To The Docker Container Where The Code Is B

Hi all, I have a quick question regarding passing secrets to a docker container. Our set up is the following: we use GCP Autoscaler to spawn "workers" where the training/testing is done. In order to pass secrets to the docker container where the code is being run, I am thinking of using Google Secrets Manager for passing secrets. However, the docker container does not have the same execution context as the VM, so what would be a safe way of querying the secrets manager for the secrets?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					PompousSpider11
				
					0
					 × 1

Votes Newest

Answers 2

Hi @<1523701087100473344:profile|SuccessfulKoala55> , I thought of passing environment parameters directly to the docker, but that approach would be a bit unsafe. Your idea makes definitely more sense...

  				
Posted 
	one year ago

					More
				  		
  Report
		
					PompousSpider11
				
					0
					 × 1

Hi @<1601023807399661568:profile|PompousSpider11> , that depends on your setup. You could inject something into each worker machine using the autoscaler startup script that will allow you to query the Google secrets manager, for example

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

Write your answer

968 Views

2 Answers

one year ago

one year ago

Tags

Similar posts

Hi All. I'M Trying To Start Docker Container In Agent Not As A Root. Read, That There Is Clearml_Agent_Exec_User Environment Variable For That. But Is Seems To Be Ignored, When I Pass It As With Docker'S -E Argument. What'S The Correct Way To Make Agent N

What The Best Practice To Configure Clearml-Agent In The Way To Don’T Install Packages That Are Already Available Every Time? I Use Nvidia-Docker To Run The Train Script For Example. I Initialized Clearml-Agent Inside A Running Nvidia-Docker Container

Hey There, I Am Trying To Use A Local Docker Agent, But I Cannot Run Tasks There. Using The Bare Metal Variant With Venvs Works Fine. It Seems As The Process Cannot Proceed After Installing Clearml-Agent On The Docker Container, Because The Experiment Is

Help Please. I Have My Clearml Server Running In A Docker Container. Now, I Am Training My Ml Models In Another Docker Container. I Want To Track These Models With My Clearml Server Located In The First Container. What Configuration Do I Need To Do?

Help Please. I Have My Clearml Server Running In A Docker Container. Now, I Am Training My Ml Models In Another Docker Container. I Want To Track These Models With My Clearml Server Located In The First Container. What Configuration Do I Need To Do?

Hi, I'M Trying To Run My Code With A Pipelinecontroller Within A Docker Container Instead Of On My Local Computer. Currently Having Trouble Where The Code Doesn'T Run In The Repo As Expected; "No Repository Found, Storing Script Code Instead" Is The Warni

Hi Everyone! I'M Having A Trouble With Executing Tasks Remotely From Web Ui Using Clearml Agent Docker Mode. (With Self-Hosted Server On Windows Computer) I Enqueued A Training Task From Web Ui, And The Docker Container Is Always "Running", But The Acutal

Hello, I'M Confused About The Best Way To Use A Docker Container To Manage The Environment For Experiments. I'Ve Gone Through Some Of The Tutorials Running In Venv Mode And Am Now Trying To Run Some Of My Own Codebase. I Have A Docker Container With All T

Hi I Have An Issue With The Clearml-Agent In Docker Mode. I Am Trying To Mount Additional Stuff In The Container Via Sur Default Via The Agent.Default_Docker.Arguments - Namely The Docker Socket. I See

Hi Everyone, Is There An Option To Not Mount Cache Into The Docker Of An Clearml-Agent? E.G. Datasets Are Downloaded Into The Docker And Gone When The Docker Container Is Removed By The Agent.