Yes, I think - however - that it is our over strict security policy, that don't let anyone to enter the repo. We're lucky that they let the developers see their code...

We have defined an SSH key for clearml, and it is also set in the /clearml-agent/.ssh/config and it still can't clone it. So it must be some security issue internally...

Sorry - my bad.
It did not work.

When I don't set anything in the clearml.conf - it took the repo from the cache. When I delete the cache, it can't get the repo any longer.

We host in a private (self-hosed) git-Lab, which can only be cloned through the SSH, and we would like to import packages by compiling it from a public git-Hub (or by installing it using wheels with find-links).

Try removing force_git_ssh_protocol: true
If you do not provide user/pass , and assuming the repo link on the task is internal, hence SSH, it should leave links as they are (ssh for ssh http for http), this should solve the issue (I hope)

I've only set the force_git_ssh_protocol to false, but kept the force_git_ssh_protocol / force_git_ssh_protocol - which are set to simply some port and 'git'. It didn't work unfortunately... could not connect to github with the port (connection timed out)

If I change or remove the port, I can't clone the whole project, so I don't even reach the installation of the detectron part.

PleasantGiraffe85

it took the repo from the cache. When I delete the cache, it can't get the repo any longer.

what error are you getting ? (are we talking about the internal repo)

Exactly - we need a mixed behavior!
We host in a private (self-hosed) git-Lab, which can only be cloned through the SSH, and we would like to import packages by compiling it from a public git-Hub (or by installing it using wheels with find-links).

After a restart, that seems to have helped, thanks! 🙂
Now we just need to solve the compilation of the git source installation... would have nice to have the find-links for the wheels, but I understand it's unstable in reproduce-ability.

If this is how the repo links look like, do not set anything in the clearml.conf
It "should" use the ssh for the ssh links, and http for the http links.

A public one is easy:
https://github.com/facebookresearch/detectron2
The internal one is something like:
ssh://repo.our-domain.com:1234

We use that ssh so that we can easily access it, without the need of storing name/password, or without the need that each one who uses the code has to set up in advance their credentials in ENV vars and such...

Hi PleasantGiraffe85
Did you set git_host to only point to your host ? do you expect all the git clones to use SSH? how does the requirements.txt git link looks like ?
https://github.com/allegroai/clearml-agent/blob/bf07b7f76d3236c1118b81730c6d9718705a795a/docs/clearml.conf#L22

This means it will Always authenticate with SSH force_git_ssh_protocol ...
But it seems you need mixed behavior ?
Are you using github as git provider ?

Hi AgitatedDove14 , thanks for the quick response.
I didn't set git_host , only force_git_ssh_protocol: true
force_git_ssh_port: ...
force_git_ssh_user: ...

If I understand correctly, git_host / git_user / git_password are all for HTTP, and we're using the SSH to clone the project through the agent.

PleasantGiraffe85 can you send examples of the different git repo links (one internal one public) ?

We're lucky that they let the developers see their code...

LOL 😄

and it is also set in the 

/clearml-agent/.ssh/config

 and it still can't clone it. So it must be some security issue internally.

Wait, are you using docker mode or venv mode ? in both cases your SSH credentials should be at the default ~/.ssh