suspect permissions, but not entirely sure what and where

Seems like it.
Check the config file on the agent machine
https://github.com/allegroai/clearml-agent/blob/822984301889327ae1a703ffdc56470ad006a951/docs/clearml.conf#L18
https://github.com/allegroai/clearml-agent/blob/822984301889327ae1a703ffdc56470ad006a951/docs/clearml.conf#L19

Thanks Martin. I think I have found where the error is!

Is there any documentation on how to set up the config for the agent?

I am struggling to fill in the values for the template. Some are obvious, others are not

Sure, run:
clearml-agent initIt is a CLI wizard to configure the initial configuration file.

Hey @<1523701205467926528:profile|AgitatedDove14> I am helping Max to get this working. I ran the clearml-agent init and now have the correct entries in the clearml.conf file.

Created an ssh key from the agent, uploaded it to the git repo, but still getting this error:

Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

If I manually ssh on to the agent, and run:
git clone git@github.com:<repo_path> it clones just fine, so that rules out any connectivity issues.

Is there anything specific in the conf file I should look at?
Is there anyway to debug this from the docker container?

Just by chance I set the SSH deploy keys to write access and now we're able to clone the repo. Why would the SSH key need write access to the repo to be able to clone?

Hi @<1687643893996195840:profile|RoundCat60> , I just saw the message,

Just by chance I set the SSH deploy keys to write access and now we're able to clone the repo. Why would the SSH key need write access to the repo to be able to clone?

Let me explain, the default use case for the agent is to use user/pass (as configured in the clearml.conf file(
It will change any ssh links to https links and will add the credentials to clone the repository.
You can also provide SSH keys (basically whatever you have on the host machine will be used by the agent)
In that case, you should make sure it changes all the repo link to ssh links, by setting:
force_git_ssh_protocol: true
None

Yep i've done all that, it didn't seem to work until I set the deploy key to write

That is a bit odd, But SSH keys have to have a specific chmod flags for them to work (security issues)
What was the error ?

Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.