Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Trying

Hi, I am trying clearml-serving with auto-update and Triton via
clearml-serving --id e79ede664316482798a315bd5962daad model auto-update --engine triton --endpoint "test_model_pytorch_auto" --preprocess "preprocess.py" --name "train pytorch model" --project "serving examples" --input-size 1 28 28 --input-name "INPUT__0" --input-type float32 --output-size -1 10 --output-name "OUTPUT__0" --output-type float32and it seems like input/output names are not propagated to Triton config file.

On Triton server, the config is
input: [{ dims: [1, 28, 28] data_type: TYPE_FP32 }] output: [{ dims: [-1, 10] data_type: TYPE_FP32 }]which is missing name in inputs and outputs .

Because of that Triton server can't load the model and ends up with:
Invalid argument: model input must specify 'name' for test_model_pytorch_auto_1
It works If I don't use model auto-update but only model add.

Any idea how to fix that? Thank you!

  
  
Posted 2 years ago
Votes Newest

Answers 9


Not sure what you mean.

If I understand the flow correclty, model auto-update creates a monitor in model_monitoring and then after model that complies with the monitor is found it is added to model_monitoring_eps with specific model version and it's config. Except that it's failing because input_name/output_name disappeared somewhere between model_monitoring and model_monitoring_eps ` .

  
  
Posted 2 years ago

Seems like I am using 1.2 according to https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-serving

And Images used in k8s cluster are:
allegroai/clearml-agent-k8s-base:1.24-18 allegroai/clearml-serving-inference:1.2.0 allegroai/clearml-serving-statistics:1.2.0 allegroai/clearml-serving-triton:1.2.0-22.07 allegroai/clearml:1.7.0

  
  
Posted 2 years ago

I see, let me check the code and get back to you, this seems indeed like an issue with the Triton configuration in the model monitoring scenario.

  
  
Posted 2 years ago

Hmm is "model_monitoring_eps" another version of the model and it does not have all the properties of the "original" one?

  
  
Posted 2 years ago

Sure, there is a Github https://github.com/allegroai/clearml/issues/821 now :)

  
  
Posted 2 years ago

model_monitoring has name but model_monitoring_eps doesn't

  
  
Posted 2 years ago

MelancholyChicken65 what's the clearml-serving you are using ? (I believe this issue was fixed in 1.2)

  
  
Posted 2 years ago

That looks like a bug, would you mind copy pasting this into a github issue? 🙂 AgitatedDove14 is there something else this could be?

  
  
Posted 2 years ago

MelancholyChicken65 found it ! thank you for finding this issue.
I'm hoping to get an update soon 🙂

  
  
Posted 2 years ago
1K Views
9 Answers
2 years ago
one year ago
Tags