Unanswered
Hey There I'M Currently Trying To Set Up Clearml Serving While Using Triton As My Engine. I'M Currently Experincing 405 Errors When Calling My Endpoint Or Triton Just Crashes. Can Anyone Help Out?
Since recieving the 405 the triton container just prints the following information Info: syncing models from main serving service
reporting metrics: relative time 13998 sec I already restarted clearml completely. This Morning the triton container just crahsed when serving a different model There I've got the following errors:
Update model v1 in /models/tritontest/1
copy model into /models/tritontest/1/model.bin
Starting server: ['tritonserver', '--model-control-mode=poll', '--model-repository=/models', '--repository-poll-secs=60.0', '--metrics-port=8002', '--allow-metrics=true', '--allow-gpu-metrics=true']
1724678179082 mips-data-analytics-platform-clearml-serving-triton-5d8874kpsq2 error Traceback (most recent call last):
File "clearml_serving/engines/triton/triton_helper.py", line 540, in <module>
main()
File "clearml_serving/engines/triton/triton_helper.py", line 532, in main
helper.maintenance_daemon(
File "clearml_serving/engines/triton/triton_helper.py", line 274, in maintenance_daemon
raise ValueError("triton-server process ended with error code {}".format(error_code))
ValueError: triton-server process ended with error code 1
37 Views
0
Answers
3 months ago
3 months ago