BTW: @<1673501397007470592:profile|RelievedDuck3> we just released 1.3.1 with better debugging, it prints full exception stack on failure to the clearml Serving Session Task.
I suggest you pull the latest image re run the docker compose and check what you have on the serving session Task in the UI

I test this: None , and I haven't encountered any error. I will test the custom example and I will provide you with feedback, thank you very much for your response.

Is it in the serve instance task console that I should check the exception stack?

Thanks to the exception stack I examined, I understood that I had a model registry issue. I had used joblib to save the model file on my system, and I believed that the model registration in ClearML storage was automatic. So when I made the API call, the model path returned NoneType. Once I fixed that, I was able to serve my model and make API calls giving prediction results. Also, thanks to your help, I understood that I needed custom serving, and I was able to modify the preprocess.py file to suit my problem. Once again, thank you very much for your help. I was able to test ClearML serving, and I greatly appreciated its simplicity and scalability. I believe it's a tool I'll adopt in my work. The next step for me is to test ClearML's monitoring features.

Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

Yeah I think it is complicated.
I would start with the example here: None
Basically what it does is create histogram over time of the values the Rest API gets. Then in graphana it visualizes those values.
Notice that the request latency / frequency are automatically logged into grafana for all the endpoints, no need to do anything specific

I've gone through the tutorial, and I've more or less understood it. I will perform a test to make sure. Thank you very much for sharing. Please, I have a question to submit to you, do you think it's a good idea to combine this monitoring with Evidently, to calculate new metrics and visualize them in Grafana?

Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

@<1523701205467926528:profile|AgitatedDove14> , Thank you very much, I will follow your recommendation.

Yes, look for the clearml serving session ID in the web UI (just go to the home screen and put the UID in the search 🙂 )

Out of curiosity, what ended up being the issue?

@<1523701205467926528:profile|AgitatedDove14> , thank you very much for your help, I was able to fix most of my bugs thanks to your recommendations

from typing import Any

import numpy as np

Notice Preprocess class Must be named "Preprocess"

class Preprocess(object):
def init(self):
# set internal state, this will be called only once. (i.e. not per request)
pass

def preprocess(self, body: dict, state: dict, collect_custom_statistics_fn=None) -> Any:
    # we expect to get two valid on the dict x0, and x1
    return [[body.get("Pregnancies", None), body.get("Glucose", None), body.get("BloodPressure", None), body.get("SkinThickness", None),
             body.get("Insulin", None), body.get("BMI", None), body.get("DiabetesPedigreeFunction", None), body.get("Age", None)], ]

def postprocess(self, data: Any, state: dict, collect_custom_statistics_fn=None) -> dict:
    # post process the data returned from the model inference engine
    # data is the return value from model.predict we will put is inside a return value as Y
    return dict(y=data.tolist() if isinstance(data, np.ndarray) else data)

Hi @<1673501397007470592:profile|RelievedDuck3> , can you share the code you used? What's the preprocess code you're using?

Here it is:

Also what's the additional p doing at the last line if the screenshot ?

And how is the endpoint registered ?

How is the endpoint rehistred: clearml-serving --id 6c9c2c38e70b41e0a63547e3c16db234 model add --engine sklearn --endpoint "best_diabetes_detection" --preprocess "/home/caleb/diabetes_clearml/preprocess.py" --model-id e7532b8017ad4a0f92d5b537401f0585

Okay that makes sense.
best_diabetes_detection is different from your example curl -X POST " None " notice best_mage_diabetes_detection` ?

This p is not in the original code.

I have other similar endpoints for testing; that's why, if not, there is no error at this level. Even with the two endpoints, I get the same error. One clarification: I built my ML model with scikit-learn pipeline and Optuna. Now, by building another simple model without Optuna and the preprocessing pipeline with scikit-learn, that is, by simply using, for example, LogisticRegression().fit(X, y), I do not encounter any error for serving with clearml-serving; the request via its endpoint gives me the prediction. Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

I will work on it and provide you with feedback. Do you have a list of monitoring metrics provided by clearml-serving?

Interesting question, should work and looks like an interesting combination, I'm curious what you come up with.
btw: grafana itself can already provide a lot of alerts for drift etc, this is basically their histogram delta feature

Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

of course it is, did you first try the example , here: None
If you need to run your own LogisticRegression call you can use this example:
None
Notice this is where the custom endpoint actually calls the prediction: None