Hi @<1858319194597101568:profile|EagerKitten89> , from my understanding the endpoints page currently only supports LLM endpoints and clearml-serving will be supported in the next clearml-serving release.
Answered
Issue With Self-Hosted Clearml & Clearml-Serving: "Model Endpoints" Page Is Empty After Deploying An Endpoint.
Issue with self-hosted ClearML & clearml-serving: "Model Endpoints" page is empty after deploying an endpoint. Hey community!
I've run into an issue with my self-hosted ClearML with clearml-serving, deployed via docker-compose according to the documentation. It seems I've hit a wall after some deep diagnostics and I'm hoping for your advice.
TL;DR: The "Model Endpoints" page is empty because the new clearml-server reads endpoint statuses from Redis, but the old allegroai/clearml-serving-inference doesn't know how to report its status there.
My Setup:
- Server:
clearml/server:latest— the newest version. - Serving:
allegroai/clearml-serving-inference:latest— the image hasn't been updated in ~16 months. - The deployment was done following the official
docker-compose.ymlexamples from the documentation: None and NoneSymptoms & What I've Checked: - ✅ The
clearml-agentis working, and theservicetask is launched. - ✅ The inference container responds to direct
curlrequests to the model. - ✅ An API call to
tasks.get_by_idfor this task successfully returns all the data (status:"in_progress", etc.). - ❗ ️ An API call to
serving.get_endpoints(which the UI uses) returns an empty array :{"data":{"endpoints":[]}}.The Cause (as I've discovered): - I analyzed the
clearml-servercode (/apiserver/bll/serving/__init__.py). - The logic for
get_endpointsdepends 100% on data in Redis (keys matchingserving_container_*). - I checked Redis with the command
SCAN 0 MATCH "serving_container_..."— it's empty .My question for you: Has anyone else encountered a similar incompatibility issue? What is the current best practice for deployingclearml-servingalongside an up-to-dateclearml-server?
It seems the public images forservinghave fallen far behind the server. Is there any recommended path forward other than downgrading the server to version0.17.5
870 Views
1
Answer
3 months ago
3 months ago
Tags