Hi folks. I am trying to set up Clear ML in our k8s cluster and would like to expose the Webserver and API server outisde our cluster. I am bit lost with the configuration settings and the ways to do it securely in k8s. I read https://clearml.slack.com/archives/CTK20V944/p1630682666183600 discussion and like smos I am also a bit shaky when it comes to web security. Is there some documentation explaining in simple terms what the different keys/secrets are used for?
“CLEARML__SECURE__HTTP__...

SuccessfulKoala55 With enableConfigVolume approach , I have to ensure that a apiserver.conf with the credentials is added to the storage volume?

AgitatedDove14 TBH, the IAM role scenario I havent tested out since I couldnt get it to work with temp credentials. I can get back to you on this!

And yes I assumed from the docs that the OS env will overwrite the config file ( which I dint provide since I set the credtials based on your answer on https://stackoverflow.com/questions/66216294/clearml-how-to-change-clearml-conf-file-in-aws-sagemaker ) . But somehow it does not work with temp credentials ( where apart from the secret access...

Hi. Its regarding boto clients inside kubernetes cluster and clients in machines of the developers.
In case of clients in the kubernetes cluster we use IAM policies attached to the serviceaccount to enable access to the s3 bucket. In this case I wonder how clearml sdk gets access to the s3 bucket if it relies on secret access key and access key id.
In case of clients in the machines of developer, we use https://github.com/Versent/saml2aws to retrieve temporary credentials. Since these are ...

AgitatedDove14 Thank you! I was wondering what this flag was about! I will test this and update here for future reference!

This works as expected! Thanks AgitatedDove14 Maybe we could add it to the documentation https://clear.ml/docs/latest/docs/integrations/storage/ ? I think its important.

Thank you for the confirmation SuccessfulKoala55

Another option is to pass these settings as environment variables, if that can suit your requirements

The issue with the environment variables is that the helm chart updated the env variables from a secret as shown https://github.com/allegroai/clearml-helm-charts/blob/e16060f2ad9f7075531e132ad9f8155dc5e7870f/charts/clearml/templates/deployment-apiserver.yaml#L80 . If I additional pass the auth information using extraEnvs I am n...

Right, basically someone needs to configure the “regular” environment variables for boto to use the IAM role,

clearml

will basically uses boto, so it should be transparent. does that make sense ? How do you spin the job on the k8s cluster and how do you configure it?

Yep I was thinking the same that the design choice must have been inspired by transparency. At the moment we just use the sdk to log training runs, model eartifacts etc and upload the model. We dont use t...