You're quite right on that point, right now you can have some sort of user credentials store next to the k8s glue, but I know the ClearML paid roadmap has plans to support this kind of user-specific server-side configuration.

Hi, scenario as follows.

client.py runs task.execute_remotely(queue='myqueue', exit_process=True) The API section of clearml.conf at client side is read in. client side calls clearml server and insert task into queue. K8S glue retrieves task from queue. Spawn a K8S pod. K8S pod performs git clone Error. ssh keys not found.
Each individual has their own key in the gitlab profile and gitlab is configured to only work via ssh.
We can't place the key in the image as this is as good as giving out the key for others to steal. We can't use the TRAINS_AGENT_GIT_USER as Gitlab is configured to only allows SSH.

It's not currently supported, but can be done by changing the k8s-glue pod-spinning code

Hey SubstantialElk6 , this will require mounting the .ssh dir as part of the pod spinning, based on the SSH secret of the user

Thanks SuccessfulKoala55 . I can try my hand on a patch. But the pod spinning is handled by the k8s glue, which has no link to the client side. How should the client pass the key over to k8s glue during runtime via clearml server?

And any roadmap on this? The organisation's on ssh auth is firm. This can end up not possible to use ClearML for remote execution.

Hi SubstantialElk6 , what exactly is missing?

Thanks SuccessfulKoala55 . Just pm'ed him.

what feature on this paid roadmap are you referring to? I am indeed communicating with Noem on paid features.

I'm not sure exactly, you'll have to ask him 🙂