In short: the issue is that during model training, ClearML installs packages from both the experimental (non-master) branch and the master branch, leading to version conflicts 🙂

@<1523701070390366208:profile|CostlyOstrich36> cool, it worked! Thanks a lot!

Aha, cool, looks like that’s what I needed! thanks @<1729671499981262848:profile|CooperativeKitten94> !

Hi @<1523701205467926528:profile|AgitatedDove14> ,
I set force_git_ssh_protocol: true (leaving git_user and git_pass empty) because I want to use ssh_key-based authentication.
The problem is that even when I mount the SSH key into the root home directory (e.g., /root/.ssh/id_rsa with the correct permissions set to 400) I still encounter the same error. I suspect that I might be doing something wrong, or perhaps the open-source version does not support this configuration. Could this ...

Hi Martin! About pip install -e . - I had that made through agent.package_manager.extra_pip_install_flags flag like below:

agent {
   package_manager: {
          extra_pip_install_flags: ["-e", "." ],
        }
}

Thanks for your help!

OK, I managed to do this via extra_docker_shell_script option in values.yaml:

clearmlConfig: |-
    sdk {
    }
    agent {
        force_git_ssh_protocol: true
        git_user: ""
        git_pass: ""
        extra_docker_shell_script: ["mkdir -p /root/.ssh", "cp /etc/ssh-key/..data/id_ed25519 /root/.ssh/", "ssh-keyscan -H bitbucket.mycompany.com >> /root/.ssh/known_hosts"]
       
    }

And now I have another issue - how can I run pip install -e . command? Is that possib...

Sorry, got sick and just now got to my laptop, I will try to find logs tomorrow (if they still there ’cause this is test environment and my teammate could already delete them).

got sick and just now got to my laptop. Yep, this works great. Thanks again for the advice, @<1729671499981262848:profile|CooperativeKitten94>

k get deployment/clearml-agent -o yaml | grep -A2 imagePullSecrets
      imagePullSecrets:
      - name: regcred
      initContainers: