Hey Martin.
I’m deploying to k8s and want to know if PVC is necessary for Redis.

I'll try that. How can I make it to use authentication too?

Hmm maybe this will work
https://elasticsearch-py.readthedocs.io/en/v7.12.1/#tls-ssl-and-authentication

I'll try that and report back

Ok thanks for reply

https://github.com/allegroai/clearml-server/blob/babfcbb70701fc725448556d1e691cdfd1292949/apiserver/es_factory.py

https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_config.html

I could not find anything from here.

Oh well it doesn’t work for me then… I keep getting errors.

How is it going to access to actual pod? Is it a headless service?

SuccessfulKoala55 I’m getting 405 on api calls with the configuration you proposed. (btw I think USER_KEY is right, with a single underscore)

yes

The K8sIntegration can be made into a nice operator for example and deeply integrate into ClearML itself. :grinning_face_with_star_eyes:

No it doesn’t work from entirely new browser too. 😞
As this is a staging instance. let me remove and deploy without these changes.

Hi AgitatedDove14

For example version control, A/B testing, shadow testing, rollback etc..
We can use conventional solutions that we already use. Like Helm and Istio.

But for example is there any tools that gives more insight into model performance in production? or gives more knobs and settings specific for AI.
using common microservice deployment frameworks will be limiting.

I found some solutions (KubeFlow, Seldon Core, http://run.ai , MLFlow, …) but haven’t got to look at them really.

Yes I see. But the current env doesn't reach there because it cannot be decoded by pyhocon.

I'm using a simple pyhocon code to test it and it won't decode. So it won't reach the code to ovverride settings at all.

no. tried many variations. I'm not sure if it's reading the variable or not. because wait times don't change either

I changed them and tried some combinations to no avail...
I assume it should change the retry pattern too. but it doesn't change.. I feel I'm missing something obvious here

Thanks!

curl works with the same url and creds from the container...

it is indeed parsing because invalid hocon leads to error. but nothing changes about retry times etc

That wouldn't have crossed my mind especially when it's not in the docs.

It's still not connecting XD
but parsing is successful 😄

aha got it..
I assume env vars would be configured according to this doc:
https://allegro.ai/clearml/docs/docs/deploying_clearml/clearml_server_config.html#dynamic-environment-variables

there's no log of connecting to elasticsearch now 😕
does it mean it is okay?

the api server doesn't come up though. readiness probe fails.

no...

ah let me try sth

of course because the tuple is valid in Python.
but there's no way I can supply a tuple in the HOCON format (of json for that matter(

yes. Tuple is not valid. like json. only scalar, string, dict or array