Hi @<1697056701116583936:profile|JealousArcticwolf24>
Awesome deployment 🤩
Yes if you need another scalable model serving you can just run another instance of the clearml-serving-inference
https://github.com/allegroai/clearml-serving/blob/7ba356efc97a6ae2159283d198d981b3c1ab85e6/docker/docker-compose.yml#L77
So you end up with two of them, one per models environment. Notice each one should have its own unique clearml serving session
https://github.com/allegroai/clearml-serving/blob/7ba356efc97a6ae2159283d198d981b3c1ab85e6/docker/docker-compose.yml#L92
@<1523701205467926528:profile|AgitatedDove14> Thank you for the answer! So i will be able to log everything in the same grafana ? and i dont need to run another docker-compose with new clearml inference ?)
@<1523701205467926528:profile|AgitatedDove14> thanks) actualy im already solved the problem) just build another docker-compose (only with clearml-inference) with external network) conected to first docker-compose) where im running whole bunch of containers with grafana and prometeus) and now im able to use 2 diff envs in 2 diff containers) but logging in the same prometeus and grafana)
@<1523701205467926528:profile|AgitatedDove14> Looks like it’s not so easy((( i run model in independent container… but cant find metrics in grafana( should i add this service into docker-compose ? or what? if i should add it into docker-compose how can i add new models without rebuild whole docker compose ? or i just need to add configs into env of my dockerfile ?
@<1523701205467926528:profile|AgitatedDove14> and should i open ports ? or maybe just add network into my new model ? can’t understand what to do(
i can get predictions from container but not able to use whole tool stack(
and its cool that we have here kafka and so on but looks like i dont use it when i just build container with another environment
i can run clearml-serving-infernece in another docker-compose and use network and environment from main clearml-serving docker-compose
looks like there should be something that i cant find to make it more finished
Hi @<1697056701116583936:profile|JealousArcticwolf24> just saw the reply
Image look okay?! what what is the query? basically I'm truing to understand if grafana is connected to the Prometheus, and if the Prometheus has any data in it
Secondly, just to make sure, kafka service should be able to connect directly to the the container running the actual inference
Let's start small. Do you have grafana enabled in your docker compose and can you login to your grafana web ui?
Notice grafana needs to access the prometheus container directly so easiest way is to have everything in the same docker compose
@<1523701205467926528:profile|AgitatedDove14> Yes) for first two models running in first env) i have) im logging outputs and inputs for this two models)
@<1523701205467926528:profile|AgitatedDove14> yes)
And can you see your promethues in your grafana?
but im not sure that it will works(