My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Answered

My autoscaled instance fails when running "git clone" on a private repo.

I do have the SSH key placed at /root/.ssh/id_rsa on the machine, and when I SSH into the machine and run sudo su; git clone <the repo> it succeeds.

Also, in the extra_vm_bash_script field: I added a whoami command which prints root , so it seems like the user being used to run the git clone during task execution is in fact root .

For context, here's the startup command that the autoscaler runs:

python -m clearml_agent --config-file /root/clearml.conf daemon --queue aws_4gpu_machines --docker python:3.9

Full log included...

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Votes Newest

Answers 34

The key seems to be placed in the expected location

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

It's an Amazon Linux AMI with the AWS CLI pre-installed on it. It uses the AWS CLI to fetch the key from AWS SSM Parameter Store. It's granted read access to that SSM Parameter via the instance role.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Let's see. The screenshots above are me running on the host, not attaching to a running container. So I believe I do want the keys to be mounted into the running containers.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Wow, it really does not want to show the output of those print statements in stdout. Here's the output of the task from the console after cloning it. Confirmed that the setup script and all code changes are present:

  				
Posted 
	one year ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Show more results

Write your answer

60K Views

34 Answers

one year ago