My Autoscaled Instance Fails When Running "Git Clone" On A Private Repo. I

Answered

My autoscaled instance fails when running "git clone" on a private repo.

I do have the SSH key placed at /root/.ssh/id_rsa on the machine, and when I SSH into the machine and run sudo su; git clone <the repo> it succeeds.

Also, in the extra_vm_bash_script field: I added a whoami command which prints root , so it seems like the user being used to run the git clone during task execution is in fact root .

For context, here's the startup command that the autoscaler runs:

python -m clearml_agent --config-file /root/clearml.conf daemon --queue aws_4gpu_machines --docker python:3.9

Full log included...

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Votes Newest

Answers 34

oh that makes sense.
I would add to your Task's docker startup script the following:

ls -la /.ssh
ls -la ~/.ssh
cat ~/.ssh/id_rsa

Let's see what you get

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

I

do

have the SSH key placed at

/root/.ssh/id_rsa

on the machine,

@<1541954607595393024:profile|BattyCrocodile47> is the SSH key part of the containers? or are you saying it is on the EC2 instance ?

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

Remove this from your startup script:

#!/bin/bash

there is no need that, it actually "markes out" the entire thing

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					AgitatedDove14
				
					0
					 × 1

So here's a snippet from my aws_autoscaler.yaml file

  				
Posted 
	2 years ago

					More
				  		
  Report
		
					BattyCrocodile47
				
					0
					 × 1

Show more results

Write your answer

178K Views

34 Answers

2 years ago

For Clearml-Agents Where Does It Clone The Git Repo And Can You Specify The Location Somehow?

Hi, I Am Wondering Is There Anyway To Use Clearml Agent Without Setting Up The Environment, For Example, Do Not Enforce The Git Clone And Pip Install But Cd Some Dir And Activate An Environment?

Hey There! Question About The Aws Autoscaler, The Tldr Is That I Can'T Get The Aws_Autoscaler.Py When Running With --Remote Flag To Clone My Git Repository (Hosted On Gitlab). Here'S What I Did So Far:

Hi In My System, Clearml Is Automatically Taking Git Hub A My Respository Even Though I Upload The Data From Local Drive... This Is Causing While I Clone The Model And Trying To Retrain With New Data/New Hyperrameter.. Can Anyone Pls Help Me Here

Hello I Am Running Code Using Git For Custom Module Inclusion, Now If I Clone The Experiment And Change The Code In It Gives Me Git Difference Error. And Will I Have To Run The Code Manually Every Time When I Do Git Commit?

I Am Trying To Run A Task That Is Completely Detached From Git - Remotely. The Script Uploads Fine But In The Ui, The Git Repo Appears As “Origin”. When The Agent Tries To Pick This Up, It Fails On Trying To Clone “Origin”. What Can I Do To Let The Agent

Another Thing That Would Be Useful Is To Enable Git Clone Via Github Personal Access Token As Ssh Config Can Be Annoying When Working In Containerised Env For Cloning Tasks In The Agent

Hi, I Was Using The K8S Glue And It Worked Fine On One Project But Didn'T Work On Another. At The Point Just Before A Git Clone Was Executed, I Get The Error

Hey, How Can I Add A Private Key In Order To Let The Clearml Agent To Clone From A Private Git Repository?