Unanswered
[Clearml-Session Discussion]
I Have Modified Clearml-Session And Clearml-Agent To Provide Gpu Resources To Our Company As A Tool, And It Is Being Well-Utilized By Data Scientists And Mlops Engineers In Our Company. Here’S What I Modified:
@<1523701205467926528:profile|AgitatedDove14> @<1529271085315395584:profile|AmusedCat74> Hi guys 🙌
- I think that by default it uses the host network so it can take care of that, are you saying you added k8s integration ?-> Yes, i modified clearml-agent helm chart.
- “SSH allows access with password” it is a very long random password, not sure I see a risk here, wdyt?-> Currently, when enqueueing a task, clearml-session generates a long random password for SSH and VS Code and displays it in the user properties(see pic). We only run the tasks when needed and turn them off when not in use. However, I understand that it may be less secure compared to using SSH tunneling. However, convenience is prioritized more in our company.
- I’m assuming this only works if you are adding an HTTP/s routing ?-> I haven’t added any HTTP/s routing myself. When creating services for SSH, VS Code, and Jupyter in the service-sessions.yaml file of the clearml-agent Helm chart, I used nodePort for each service. Later, when enqueueing tasks, the clearml-session script is aware of these nodePorts and generates the corresponding URLs.
I must admit that this approach is somewhat hardcoded. I wanted to use Istio, similar to Kubeflow Notebook, to create URLs like “<public ip>/jupyter/?ns=<namespace>“. So, as I’m planning to enhance the system, I thought it would be good to get your opinion in advance and consider submitting a PR to your side.
162 Views
0
Answers
one year ago
one year ago