Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hey Everyone, I'M Having An Issue Due To Conflicting Git Credentials On The Clearml-Agent (Running Inside The Docker). I'M Using Ssh Settings (

Hey everyone,
I'm having an issue due to conflicting git credentials on the clearml-agent (running inside the docker).
I'm using SSH settings ( git_ssh_port/user ) to clone the project from our internal git, but also have in the requirements.txt a reference to detectron2 from the github installation. The Task or Pipeline step fails to install the detectron2 from source (github). I tried switching to wheel installation using a specific location with find-links specification in the requirements.txt , but that didn't work either.
ERROR: No matching distribution found for detectron2==0.6+cu111
Any ideas?

  
  
Posted 2 years ago
Votes Newest

Answers 14


Hi PleasantGiraffe85
Did you set git_host to only point to your host ? do you expect all the git clones to use SSH? how does the requirements.txt git link looks like ?
https://github.com/allegroai/clearml-agent/blob/bf07b7f76d3236c1118b81730c6d9718705a795a/docs/clearml.conf#L22

  
  
Posted 2 years ago

This means it will Always authenticate with SSH force_git_ssh_protocol ...
But it seems you need mixed behavior ?
Are you using github as git provider ?

  
  
Posted 2 years ago

We host in a private (self-hosed) git-Lab, which can only be cloned through the SSH, and we would like to import packages by compiling it from a public git-Hub (or by installing it using wheels with find-links).

Try removing force_git_ssh_protocol: true
If you do not provide user/pass , and assuming the repo link on the task is internal, hence SSH, it should leave links as they are (ssh for ssh http for http), this should solve the issue (I hope)

  
  
Posted 2 years ago

After a restart, that seems to have helped, thanks! 🙂
Now we just need to solve the compilation of the git source installation... would have nice to have the find-links for the wheels, but I understand it's unstable in reproduce-ability.

  
  
Posted 2 years ago

If this is how the repo links look like, do not set anything in the clearml.conf
It "should" use the ssh for the ssh links, and http for the http links.

  
  
Posted 2 years ago

We're lucky that they let the developers see their code...

LOL 😄

and it is also set in the 

/clearml-agent/.ssh/config

 and it still can't clone it. So it must be some security issue internally.

Wait, are you using docker mode or venv mode ? in both cases your SSH credentials should be at the default ~/.ssh

  
  
Posted 2 years ago

A public one is easy:
https://github.com/facebookresearch/detectron2
The internal one is something like:
ssh://repo.our-domain.com:1234

We use that ssh so that we can easily access it, without the need of storing name/password, or without the need that each one who uses the code has to set up in advance their credentials in ENV vars and such...

  
  
Posted 2 years ago

Sorry - my bad.
It did not work.

When I don't set anything in the clearml.conf - it took the repo from the cache. When I delete the cache, it can't get the repo any longer.

  
  
Posted 2 years ago

I've only set the force_git_ssh_protocol to false, but kept the force_git_ssh_protocol / force_git_ssh_protocol - which are set to simply some port and 'git'. It didn't work unfortunately... could not connect to github with the port (connection timed out)

If I change or remove the port, I can't clone the whole project, so I don't even reach the installation of the detectron part.

  
  
Posted 2 years ago

Exactly - we need a mixed behavior!
We host in a private (self-hosed) git-Lab, which can only be cloned through the SSH, and we would like to import packages by compiling it from a public git-Hub (or by installing it using wheels with find-links).

  
  
Posted 2 years ago

Yes, I think - however - that it is our over strict security policy, that don't let anyone to enter the repo. We're lucky that they let the developers see their code...

We have defined an SSH key for clearml, and it is also set in the /clearml-agent/.ssh/config and it still can't clone it. So it must be some security issue internally...

  
  
Posted 2 years ago

Hi AgitatedDove14 , thanks for the quick response.
I didn't set git_host , only force_git_ssh_protocol: true
force_git_ssh_port: ...
force_git_ssh_user: ...

If I understand correctly, git_host / git_user / git_password are all for HTTP, and we're using the SSH to clone the project through the agent.

  
  
Posted 2 years ago

PleasantGiraffe85 can you send examples of the different git repo links (one internal one public) ?

  
  
Posted 2 years ago

PleasantGiraffe85

it took the repo from the cache. When I delete the cache, it can't get the repo any longer.

what error are you getting ? (are we talking about the internal repo)

  
  
Posted 2 years ago