we’re running it with the older helm chart if that matters. anyways I can’t see anything related to Gunicorn in chart or configs.

no. apparently changing these variables causes something to fail.

SuccessfulKoala55 Hi Jake
We didn’t change anything related to gunicorn. Is there any specific thing I can check for?
Also I noticied that it’s not running the gunicorn as a command but loads it in the python code, I don’t think it’s possible to change the threading with env that way.

ah ok. So only solution is changing the hardcoded one…
I will be looking into docs to find the right env variable. If you know it I’d appreciate you telling me 😄 SuccessfulKoala55

Awesome info! Thanks!
What env variable should I use?
Also maybe I can block unauthenticated access to that endpoint too, I guess clearml-agent doesn’t use it!

The login prompt doesn’t appear (I tried removing cookies and all too)

SuccessfulKoala55 I’m getting 405 on api calls with the configuration you proposed. (btw I think USER_KEY is right, with a single underscore)

CLEARML__SECURE__HTTP__SESSION_SECRET__APISERVER
Does this one need to be changed?

The dashboard doesn’t load

No it doesn’t work from entirely new browser too. 😞
As this is a staging instance. let me remove and deploy without these changes.

webserver deployment:

Thanks a million SuccessfulKoala55 😍

Ok I think I might have found the problem.

Huh it works now…!

It’s probably unrelated 😬 I’ll keep you posted.

yep it was unrelated.. sorry
Thanks for your help

Hey Martin.
I’m deploying to k8s and want to know if PVC is necessary for Redis.

oh great!
let me take a look at this. There’s a lot of custom tooling so integration is very important.

cc: FiercePenguin76 ^^^

I actually got the idea to disable istio for the container. I think that'll work considering api-server doesn't need to be accessed directly from outside.

I'll test that and then your approach. If nothing works then I'll post again 😄

Thanks for your help Jake

hmm
that would be great.

Yes.
ClearML has all these awesome dashboards. I was thinking maybe there will be some deploy to prod/staging/shadow button so you can do everything from there. (Like Gitlab CI/CD if you have seen that)
That would be ideal.

I want to find a good deployment framework (if exists) and integrate it into our ClearML workflow).
like get latest release with prod tag from clearml api and apply that to k8s.
something along those lines

aha got it..
I assume env vars would be configured according to this doc:
https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_config.html#dynamic-environment-variables

standard URI
according to:
https://elasticsearch-py.readthedocs.io/en/v7.12.1/#tls-ssl-and-authentication

Hmm maybe this will work
https://elasticsearch-py.readthedocs.io/en/v7.12.1/#tls-ssl-and-authentication

I'll try that and report back

https://github.com/allegroai/clearml-server/blob/babfcbb70701fc725448556d1e691cdfd1292949/apiserver/es_factory.py

https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_config.html

I’m using cloud-ready helm chart (with some modifications)
So for api-server deployment:

Ah I think I understand it now. 🙂
There’s a static number of pod which services are created for…

How is it going to access to actual pod? Is it a headless service?

I’m not using templates for k8s glue. I’m using the default operation mode which uses kubectl run. Should I use templates and specify a service in there to be able to connect to the pods?