Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Am Trying

Hi, I am trying clearml-serving with auto-update and Triton via
clearml-serving --id e79ede664316482798a315bd5962daad model auto-update --engine triton --endpoint "test_model_pytorch_auto" --preprocess "preprocess.py" --name "train pytorch model" --project "serving examples" --input-size 1 28 28 --input-name "INPUT__0" --input-type float32 --output-size -1 10 --output-name "OUTPUT__0" --output-type float32and it seems like input/output names are not propagated to Triton config file.

On Triton server, the config is
input: [{ dims: [1, 28, 28] data_type: TYPE_FP32 }] output: [{ dims: [-1, 10] data_type: TYPE_FP32 }]which is missing name in inputs and outputs .

Because of that Triton server can't load the model and ends up with:
Invalid argument: model input must specify 'name' for test_model_pytorch_auto_1
It works If I don't use model auto-update but only model add.

Any idea how to fix that? Thank you!

  
  
Posted one year ago
Votes Newest

Answers 9


Seems like I am using 1.2 according to https://github.com/allegroai/clearml-helm-charts/tree/main/charts/clearml-serving

And Images used in k8s cluster are:
allegroai/clearml-agent-k8s-base:1.24-18 allegroai/clearml-serving-inference:1.2.0 allegroai/clearml-serving-statistics:1.2.0 allegroai/clearml-serving-triton:1.2.0-22.07 allegroai/clearml:1.7.0

  
  
Posted one year ago

Hmm is "model_monitoring_eps" another version of the model and it does not have all the properties of the "original" one?

  
  
Posted one year ago

MelancholyChicken65 what's the clearml-serving you are using ? (I believe this issue was fixed in 1.2)

  
  
Posted one year ago

Not sure what you mean.

If I understand the flow correclty, model auto-update creates a monitor in model_monitoring and then after model that complies with the monitor is found it is added to model_monitoring_eps with specific model version and it's config. Except that it's failing because input_name/output_name disappeared somewhere between model_monitoring and model_monitoring_eps ` .

  
  
Posted one year ago

I see, let me check the code and get back to you, this seems indeed like an issue with the Triton configuration in the model monitoring scenario.

  
  
Posted one year ago

MelancholyChicken65 found it ! thank you for finding this issue.
I'm hoping to get an update soon 🙂

  
  
Posted one year ago

Sure, there is a Github https://github.com/allegroai/clearml/issues/821 now :)

  
  
Posted one year ago

model_monitoring has name but model_monitoring_eps doesn't

  
  
Posted one year ago

That looks like a bug, would you mind copy pasting this into a github issue? 🙂 AgitatedDove14 is there something else this could be?

  
  
Posted one year ago
646 Views
9 Answers
one year ago
one year ago
Tags