Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Yet Again I Come With A Problem I Cant See A Fix For This Issue That Is Bugging Me For Days. I Want To Serve A Gpt2 Model And I Have The Onnx Uploaded To The Server. When I Try To Mount The Endpoint The Server Will Try To Find Model.Onnx As It Is Int

Hi, yet again i come with a problem i cant see a fix for this issue that is bugging me for days.

I want to serve a Gpt2 model and i have the onnx uploaded to the server. When i try to mount the endpoint the server will try to find model.onnx as it is intended but it will not find since it got renamed to model.bin as moment of the service starting but if i go

clearml-serving --id 12e416036c4b4cd38b9fd3a46c85a583 model add --engine triton --endpoint "gpt2_onnx" --model-id 75159e2de62142fb9958e416807e3d1a --preprocess preprocess.py --input-size "[-1, -1]" "[-1, -1]" "[-1, -1]" --input-type int64 int64 int64 --input-name "input_ids" "attention_mask" "token_type_ids" --output-size "[-1, -1, 50257]" --output-type float32 --output-name "logits" --aux-config platform=\"onnxruntime_onnx\" max_batch_size=8 default_model_filename=\"model.bin\"

i will get an error "Error: ERROR: You have default_model_filename in your config pbtxt, please remove it. It will be added automatically by the system."

Is there any fix for this?

Thanks again

  
  
Posted 12 days ago
Votes Newest

Answers


Following up on this i was unable to fix the issue. But i ended up finding another complication. When uploading a onnx model using the upload command it keeps getting tagged as a TensorFlow model, even with the correct file structure, and that leads to the previous issue since the serving module will search for different format than the onnx.

As far as i could see this comes from the helper inside the triton engine, but as of right now i could not fix it.

Is there anything i might be doing wrong?

  
  
Posted 11 days ago
69 Views
1 Answer
12 days ago
10 days ago
Tags