When we run a script containing Task.init from within our repo, it creates a repo URL that looks like this:
Now the agents trying to execute this task fail with:cloning: agent_user@git.mycompany.com: Permission denied (publickey). fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists.
An easy fix is to change the URL in the UI to include the user, e.g. ssh://git@git.mycompany.com:2022/myuser/repo.git , but as mentioned by AgitatedDove14 https://github.com/allegroai/clearml-agent/issues/42#issuecomment-757045256 , this does not scale well.
Using a config
I also tried creating a config, like:Host git.mycompany.com HostName 123.123.123.123 IdentityFile ~/.ssh/id_rsa IdentitiesOnly yes User git Port 2022Running the clone command outside of the docker container ( git clone ssh://git.mycompany.com:2022/myuser/repo.git ) work with this - but within the docker container, SSH refuses to use the config because the owner of the config does not match ( agent_user vs root ).
I see many potential ways to solve this, but I am not sure what is the best. Maybe I am also missing an obvious solution?
Ideally, we could set the repo user in the config on developer machines when we create the task, thereby having the correct URL directly stored in the backend. Alternatively, could we use Task.update() or something to update the repo after the call to Task.init() ? Would that propagate into the backend? And would you have an example snippet? Could the option agent.force_git_ssh_user = "git" be extended from the use-case of rewriting https to ssh URLs to also rewrite ssh URLs?
Any input would be greatly appreciated! 🙂
Found a https://github.com/allegroai/clearml-agent/issues/42#issuecomment-887331420 . Though would any of the above proposed solutions be feasible?
Hey AgitatedDove14 - thank you for the help! 🙂 Though in our case, most developers have the repo setup with ssh key authentication. Thus the task gets a 'ssh url' like ssh:// and not https:// . Consequently, the conversion is never called. Or is it already expected behavior that ClearML agent rewrites ssh://mydomain.com:2022/ ... to ssh://git@mydomain.com:2022/ ... if I have force_git_ssh_protocol: true and force_git_ssh_user: "git" ?
generally speaking the agent will convert the repo url to the auth scheme it is configured with, ssh->hhtp if using user/pass, and http->ssh if using ssh
Hm I tried it again (even cleaning up the vcs cache before since that caused an issue before) but it still does not work. Looking at the code, I also could not find the place where this should happen. For all I can tell, there are only translations from https->ssh and ssh->https , but not ssh->ssh .
To add that, I quickly coded up this PR:
https://github.com/allegroai/clearml-agent/pull/72
Could you take a look at it? On our installation here, it shows the desired behavior, I can later see in the logUsing SSH credentials - ssh url ' ' with ssh url ' 'and the cloning works 🙂
Hi EagerOtter28
I think the replacement should happen here:
https://github.com/allegroai/clearml-agent/blob/42606d9247afbbd510dc93eeee966ddf34bb0312/clearml_agent/helper/repo.py#L277
My only point is, if we have no force_git_ssh_port or force_git_ssh_user we should not touch the SSH link (i.e. less chance of us messing with the original URL if no one asked us to)
Ah OK 🤔 So should I maybe update the PR to not touch the URL if neither user nor port are 'force-set'?
Hi EagerOtter28
The agent knows how to do the http->ssh conversion on the fly, in your cleaml.conf (on the agent's machine) set force_git_ssh_protocol: true
https://github.com/allegroai/clearml-agent/blob/42606d9247afbbd510dc93eeee966ddf34bb0312/docs/clearml.conf#L25
Ohh I see now the force SSH did not replace the user in the SSH link (only if the original was http), right ?
Or is it already expected behavior that ClearML agent rewrites ...
Yep, that should work