Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

Yeah I think it is complicated.
I would start with the example here: None
Basically what it does is create histogram over time of the values the Rest API gets. Then in graphana it visualizes those values.
Notice that the request latency / frequency are automatically logged into grafana for all the endpoints, no need to do anything specific

I test this: None , and I haven't encountered any error. I will test the custom example and I will provide you with feedback, thank you very much for your response.

Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

I will work on it and provide you with feedback. Do you have a list of monitoring metrics provided by clearml-serving?

Interesting question, should work and looks like an interesting combination, I'm curious what you come up with.
btw: grafana itself can already provide a lot of alerts for drift etc, this is basically their histogram delta feature

Thanks to the exception stack I examined, I understood that I had a model registry issue. I had used joblib to save the model file on my system, and I believed that the model registration in ClearML storage was automatic. So when I made the API call, the model path returned NoneType. Once I fixed that, I was able to serve my model and make API calls giving prediction results. Also, thanks to your help, I understood that I needed custom serving, and I was able to modify the preprocess.py file to suit my problem. Once again, thank you very much for your help. I was able to test ClearML serving, and I greatly appreciated its simplicity and scalability. I believe it's a tool I'll adopt in my work. The next step for me is to test ClearML's monitoring features.

How is the endpoint rehistred: clearml-serving --id 6c9c2c38e70b41e0a63547e3c16db234 model add --engine sklearn --endpoint "best_diabetes_detection" --preprocess "/home/caleb/diabetes_clearml/preprocess.py" --model-id e7532b8017ad4a0f92d5b537401f0585

This p is not in the original code.

I have other similar endpoints for testing; that's why, if not, there is no error at this level. Even with the two endpoints, I get the same error. One clarification: I built my ML model with scikit-learn pipeline and Optuna. Now, by building another simple model without Optuna and the preprocessing pipeline with scikit-learn, that is, by simply using, for example, LogisticRegression().fit(X, y), I do not encounter any error for serving with clearml-serving; the request via its endpoint gives me the prediction. Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

I've gone through the tutorial, and I've more or less understood it. I will perform a test to make sure. Thank you very much for sharing. Please, I have a question to submit to you, do you think it's a good idea to combine this monitoring with Evidently, to calculate new metrics and visualize them in Grafana?

Is it in the serve instance task console that I should check the exception stack?

Okay that makes sense.
best_diabetes_detection is different from your example curl -X POST " None " notice best_mage_diabetes_detection` ?

Also what's the additional p doing at the last line if the screenshot ?

And how is the endpoint registered ?

BTW: @<1673501397007470592:profile|RelievedDuck3> we just released 1.3.1 with better debugging, it prints full exception stack on failure to the clearml Serving Session Task.
I suggest you pull the latest image re run the docker compose and check what you have on the serving session Task in the UI

@<1523701205467926528:profile|AgitatedDove14> , Thank you very much, I will follow your recommendation.

Hi @<1673501397007470592:profile|RelievedDuck3> , can you share the code you used? What's the preprocess code you're using?

Here it is:

Out of curiosity, what ended up being the issue?

@<1523701205467926528:profile|AgitatedDove14> , thank you very much for your help, I was able to fix most of my bugs thanks to your recommendations

Yes, look for the clearml serving session ID in the web UI (just go to the home screen and put the UID in the search 🙂 )

Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

of course it is, did you first try the example , here: None
If you need to run your own LogisticRegression call you can use this example:
None
Notice this is where the custom endpoint actually calls the prediction: None

from typing import Any

import numpy as np

Notice Preprocess class Must be named "Preprocess"

class Preprocess(object):
def init(self):
# set internal state, this will be called only once. (i.e. not per request)
pass

def preprocess(self, body: dict, state: dict, collect_custom_statistics_fn=None) -> Any:
    # we expect to get two valid on the dict x0, and x1
    return [[body.get("Pregnancies", None), body.get("Glucose", None), body.get("BloodPressure", None), body.get("SkinThickness", None),
             body.get("Insulin", None), body.get("BMI", None), body.get("DiabetesPedigreeFunction", None), body.get("Age", None)], ]

def postprocess(self, data: Any, state: dict, collect_custom_statistics_fn=None) -> dict:
    # post process the data returned from the model inference engine
    # data is the return value from model.predict we will put is inside a return value as Y
    return dict(y=data.tolist() if isinstance(data, np.ndarray) else data)