Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello Everyone, I'M Currently Trying Clearml-Serving To Serve A Model Via An Endpoint. I Followed The Tutorial In The Documentation, But When I Try A Request, I Get An Error. Here It Is: Curl -X Post "

Hello everyone, I'm currently trying clearml-serving to serve a model via an endpoint. I followed the tutorial in the documentation, but when I try a request, I get an error. Here it is: curl -X POST " None " -H "accept: application/json" -H "Content-Type: application/json" -d '{"Pregnancies": 6, "Glucose": 148, "BloodPressure": 72, "SkinThickness": 35, "Insulin": 0, "BMI": 33.6, "DiabetesPedigreeFunction": 0.627,"Age": 50}'
{"detail":"Error processing request: expected str, bytes or os.PathLike object, not NoneType"}. How can I debug this?

  
  
Posted one month ago
Votes Newest

Answers 24


from typing import Any

import numpy as np

Notice Preprocess class Must be named "Preprocess"

class Preprocess(object):
def init(self):
# set internal state, this will be called only once. (i.e. not per request)
pass

def preprocess(self, body: dict, state: dict, collect_custom_statistics_fn=None) -> Any:
    # we expect to get two valid on the dict x0, and x1
    return [[body.get("Pregnancies", None), body.get("Glucose", None), body.get("BloodPressure", None), body.get("SkinThickness", None),
             body.get("Insulin", None), body.get("BMI", None), body.get("DiabetesPedigreeFunction", None), body.get("Age", None)], ]

def postprocess(self, data: Any, state: dict, collect_custom_statistics_fn=None) -> dict:
    # post process the data returned from the model inference engine
    # data is the return value from model.predict we will put is inside a return value as Y
    return dict(y=data.tolist() if isinstance(data, np.ndarray) else data)
  
  
Posted one month ago

Hi @<1673501397007470592:profile|RelievedDuck3> , can you share the code you used? What's the preprocess code you're using?

  
  
Posted one month ago

Here it is:

  
  
Posted one month ago

BTW: @<1673501397007470592:profile|RelievedDuck3> we just released 1.3.1 with better debugging, it prints full exception stack on failure to the clearml Serving Session Task.
I suggest you pull the latest image re run the docker compose and check what you have on the serving session Task in the UI

  
  
Posted one month ago

I test this: None , and I haven't encountered any error. I will test the custom example and I will provide you with feedback, thank you very much for your response.

  
  
Posted one month ago

Yes, look for the clearml serving session ID in the web UI (just go to the home screen and put the UID in the search 🙂 )

  
  
Posted one month ago

Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

of course it is, did you first try the example , here: None
If you need to run your own LogisticRegression call you can use this example:
None
Notice this is where the custom endpoint actually calls the prediction: None

  
  
Posted one month ago

@<1523701205467926528:profile|AgitatedDove14> , Thank you very much, I will follow your recommendation.

  
  
Posted one month ago

Is it in the serve instance task console that I should check the exception stack?

  
  
Posted one month ago

I've gone through the tutorial, and I've more or less understood it. I will perform a test to make sure. Thank you very much for sharing. Please, I have a question to submit to you, do you think it's a good idea to combine this monitoring with Evidently, to calculate new metrics and visualize them in Grafana?

  
  
Posted one month ago

Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

  
  
Posted one month ago

Out of curiosity, what ended up being the issue?

  
  
Posted one month ago

Interesting question, should work and looks like an interesting combination, I'm curious what you come up with.
btw: grafana itself can already provide a lot of alerts for drift etc, this is basically their histogram delta feature

  
  
Posted one month ago

I will work on it and provide you with feedback. Do you have a list of monitoring metrics provided by clearml-serving?

  
  
Posted one month ago

image

  
  
Posted one month ago

Do you have any advice for this step, (monitoring)? I feel like it's not very well documented.

Yeah I think it is complicated.
I would start with the example here: None
Basically what it does is create histogram over time of the values the Rest API gets. Then in graphana it visualizes those values.
Notice that the request latency / frequency are automatically logged into grafana for all the endpoints, no need to do anything specific

  
  
Posted one month ago

And how is the endpoint registered ?

  
  
Posted one month ago

How is the endpoint rehistred: clearml-serving --id 6c9c2c38e70b41e0a63547e3c16db234 model add --engine sklearn --endpoint "best_diabetes_detection" --preprocess "/home/caleb/diabetes_clearml/preprocess.py" --model-id e7532b8017ad4a0f92d5b537401f0585

  
  
Posted one month ago

Also what's the additional p doing at the last line if the screenshot ?

  
  
Posted one month ago

This p is not in the original code.

  
  
Posted one month ago

Okay that makes sense.
best_diabetes_detection is different from your example curl -X POST " None " notice best_mage_diabetes_detection` ?

  
  
Posted one month ago

I have other similar endpoints for testing; that's why, if not, there is no error at this level. Even with the two endpoints, I get the same error. One clarification: I built my ML model with scikit-learn pipeline and Optuna. Now, by building another simple model without Optuna and the preprocessing pipeline with scikit-learn, that is, by simply using, for example, LogisticRegression().fit(X, y), I do not encounter any error for serving with clearml-serving; the request via its endpoint gives me the prediction. Is it not possible to serve a model with preprocessing pipeline from scikit-learn using clearml-serving?

  
  
Posted one month ago

@<1523701205467926528:profile|AgitatedDove14> , thank you very much for your help, I was able to fix most of my bugs thanks to your recommendations

  
  
Posted one month ago

Thanks to the exception stack I examined, I understood that I had a model registry issue. I had used joblib to save the model file on my system, and I believed that the model registration in ClearML storage was automatic. So when I made the API call, the model path returned NoneType. Once I fixed that, I was able to serve my model and make API calls giving prediction results. Also, thanks to your help, I understood that I needed custom serving, and I was able to modify the preprocess.py file to suit my problem. Once again, thank you very much for your help. I was able to test ClearML serving, and I greatly appreciated its simplicity and scalability. I believe it's a tool I'll adopt in my work. The next step for me is to test ClearML's monitoring features.

  
  
Posted one month ago