Yea, but before in my original setup the config file was filled. I just added some lines to the config and now the error is back.

The docker run command of the agent includes '-v', '/tmp/clearml_agent.ssh.8owl7uf2:/root/.ssh' and the file permissions are exactly the same.

In the beginning my config file was not empty 😕

(I think it is the empty config file)

Anyways, from my google search it seems that this is not something that is intuitive to fix.
Is there any progress on this: https://github.com/allegroai/clearml-agent/issues/45 ? This works on all my machines 🙂

Yes

So just tried again and still it does not work.
This is what is in .ssh on my clearml-agent
-rw------- 1 tim tim 1,5K Apr 8 14:28 authorized_keys -rw-rw-r-- 1 tim tim 208 Apr 29 11:15 config -rw------- 1 tim tim 432 Apr 8 14:53 id_ed25519 -rw-r--r-- 1 tim tim 119 Apr 8 14:53 id_ed25519.pub -rw------- 1 tim tim 432 Apr 29 11:16 id_gitlab -rw-r--r-- 1 tim tim 119 Apr 29 11:25 id_gitlab.pub -rw-rw-r-- 1 tim tim 3,1K Apr 29 11:33 known_hosts

could you remove it and test ?

However, I have not yet found a flexible solution other than ssh-agent forwarding.

And is it working?

and config is empty ?

For everyone who had the patience to read through everything, here is my solution to make clearml work with ssh-agent forwarding in the current version:
Start and ssh-agent Add ssh keys with ssh-add to agent echo $SSH_AUTH_SOCK and paste into clearml.conf as here: https://github.com/allegroai/clearml-agent/issues/45#issuecomment-779302144 (replace $SSH_AUTH_SOCKET with actually value) Move all the files except known_hosts out of ~/.ssh of the clearml-agent workstation. Start the clearml-agent.

hurray

Now it works.

size 0

Yes (Mine isn't and it is working 🙂 )

Maybe the problem is that I do not start my docker containers from the root user, so 1001 is a mapping inside the docker to my actual user. Could it be that on the host the owner if your .ssh files is called root ?

As in if it was not empty it would work?

AgitatedDove14 fyi I think this is the issue I have: https://stackoverflow.com/a/65526944/3038183

Nothing changes, still bad owner or permissions.

-rw------- 1 1000 1000 0 Feb 28 23:41 config

Yea. Not using the config file does not seem like a good long-term solution for me. However, I still have no idea, why this error happens. But enough for today. Thank you a lot for your help!

that was an odd error

👍

So it seems to be definitely a problem with docker and not with clearml. However, I do not get, why it works for you but on none of my machine (all Ubuntu 20.04 with docker 20.10)

Can you clone the git with the .ssh credentials on the host machine ?
If so, can you do the same manually inside a docker (i.e. spin a docker with mount -v /home/hostuser/.ssh:/root/.ssh) ?

btw: With the ssh agent forwarding I do not have any issues ( https://github.com/allegroai/clearml-agent/issues/45 )

agent-forwarding is working just like your described here: https://github.com/allegroai/clearml-agent/issues/45 Looking forward to not having to use the absolute path in the future 🙂

When I change the owner and the group of the files to root it works.

However, I have not yet found a flexible solution other than ssh-agent forwarding.

I ran
docker run -it -v /home/hostuser/.ssh/:/root/.ssh ubuntu:18.04but cloning does not work and this is what ls -lah /root/.ssh gives inside the docker container:
-rw------- 1 1001 1001 1.5K Apr 8 12:28 authorized_keys -rw-rw-r-- 1 1001 1001 208 Apr 29 09:15 config -rw------- 1 1001 1001 432 Apr 8 12:53 id_ed25519 -rw-r--r-- 1 1001 1001 119 Apr 8 12:53 id_ed25519.pub -rw------- 1 1001 1001 432 Apr 29 09:16 id_gitlab -rw-r--r-- 1 1001 1001 119 Apr 29 09:25 id_gitlab.pub -rw-rw-r-- 1 1001 1001 3.1K Apr 29 09:33 known_hosts