Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Unanswered
Hi All! I Recently Started Working With Clearml Serving. I Got This Example Working


Hi @<1523701118159294464:profile|ExasperatedCrab78> , I have a sad update on this issue. It does not seem to be completely solved yet. 😕 But I think I can at least describe it a bit better now:

  • Models which are located on the clearML servers (created by Task.init(..., output_uri=True) ) still run perfectly.
  • Models which are located on azure blob storage make different problems in different scenarios (which made me think we resolved this issue):- When I start the docker container add a model from the clearML server and afterwards add a model located on azure (on the same endpoint) I get no error and all my http requests are answered properly.
  • When I start the docker container with no model added and first add a model from azure, I get this error poll failed for model directory 'test_model_pytorch': failed to open text file for read /models/test_model_pytorch/config.pbtxt: No such file or directory .
  • When I start a docker container where a model from azure was already added before I get this error:
clearml-serving-triton        | Updating local model folder: /models
clearml-serving-triton        | Error retrieving model ID ca186e8440b84049971a0b623df36783 []
clearml-serving-triton        | Starting server: ['tritonserver', '--model-control-mode=poll', '--model-repository=/models', '--repository-poll-secs=60.0', '--metrics-port=8002', '--allow-metrics=true', '--allow-gpu-metrics=true']
clearml-serving-triton        | Traceback (most recent call last):
clearml-serving-triton        |   File "clearml_serving/engines/triton/triton_helper.py", line 540, in <module>
clearml-serving-triton        |     main()
clearml-serving-triton        |   File "clearml_serving/engines/triton/triton_helper.py", line 532, in main
clearml-serving-triton        |     helper.maintenance_daemon(
clearml-serving-triton        |   File "clearml_serving/engines/triton/triton_helper.py", line 274, in maintenance_daemon
clearml-serving-triton        |     raise ValueError("triton-server process ended with error code {}".format(error_code))
clearml-serving-triton        | ValueError: triton-server process ended with error code 1

Side note: In the meantime I also set up the docker container on a linux server and get the same error as on my windows computer with docker desktop.

I am not sure if this is really about passing the azure credentials because I feel like I have tried all possibilities which are suggested online. For a final try I directly wrote my azure account and storage key into the docker-compose-triton.yml with this syntax: AZURE_STORAGE_ACCOUNT: $(AZURE_STORAGE_ACCOUNT:-myazureaccount} At least this should work, or am I wrong? I would really appreciate hearing your thoughts on this.

  
  
Posted one year ago
181 Views
0 Answers
one year ago
one year ago