maybe I am entirely wrong .. I read clearml-session as running a remote 'headless container with jupyter'.. but not on k8s.. or rather... the pre-requisites say that 'ssh access', which you won't have in a k8s cluster (I would hope). I mean.. exposing the ssh port on a k8s cluster is jst a security nightmare.
Hey Manoj, I am not sure how clearml-session would know how to setup kube-proxy, if that's your intent.
Personally, I would run the clearml-server and agents on the k8s cluster, and then expose the server endpoints via kube proxy or some other nicer ingress. Then you can run jupyter locally and you should be good. Jupyter session remotely running on k8s would be a logistical nightmare 🙂
Have to get glue setup, which I couldn’t understand fully, so that’s a different topic
I suggest using the apply template setup (basically you provide a Job/Service template, and it uses that to setup k8s jobs based on the Tasks coming in from the specific queue)
Long story short, clearml-session fully supports k8s (using k8s glue)
The --remote-gateway along side ports mode will basically allow you to setup a k8s service so that every session will register with a specific port so k8s does ingest foe you and route the SSH connection to the pod itslef, everything else is tunneled over the original SSH connection.
Make sense ?