Unanswered
Hi, I Wanted To Try Model Versioning, Suppose That I'Ve A Model And Want To Have Multiple Versions Of The Same Model And To Be Able To Have Inference On These Models(For Example
Yes I'm sure that the Triton container finished syncing.
here is the Triton logs:
I0223 15:58:32.515979 71 model_repository_manager.cc:1352] successfully loaded 'yolo_2' version 1
I0223 15:58:32.842511 71 model_repository_manager.cc:1352] successfully loaded 'yolo_1' version 1
I0223 15:58:32.842579 71 server.cc:559]
+------------------+------+
| Repository Agent | Path |
+------------------+------+
+------------------+------+
I0223 15:58:32.842606 71 server.cc:586]
+-------------+-----------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Backend | Path | Config |
+-------------+-----------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
| onnxruntime | /opt/tritonserver/backends/onnxruntime/libtriton_onnxruntime.so | {"cmdline":{"auto-complete-config":"true","min-compute-capability":"6.000000","backend-directory":"/opt/tritonserver/backends","default-max-batch-size":"4"}} |
+-------------+-----------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+
I0223 15:58:32.842629 71 server.cc:629]
+---------+---------+--------+
| Model | Version | Status |
+---------+---------+--------+
| yolo_1 | 1 | READY |
| yolo_2 | 1 | READY |
+---------+---------+--------+
I0223 15:58:32.869848 71 metrics.cc:650] Collecting metrics for GPU 0: NVIDIA GeForce RTX 3080
I0223 15:58:32.871196 71 tritonserver.cc:2176]
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Option | Value |
+----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| server_id | triton |
| server_version | 2.24.0 |
| server_extensions | classification sequence model_repository model_repository(unload_dependents) schedule_policy model_configuration system_shared_memory cuda_shared_memory binary_tensor_data statistics trace |
| model_repository_path[0] | /models |
| model_control_mode | MODE_POLL |
| strict_model_config | 0 |
| rate_limit | OFF |
| pinned_memory_pool_byte_size | 268435456 |
| cuda_memory_pool_byte_size{0} | 67108864 |
| response_cache_byte_size | 0 |
| min_supported_compute_capability | 6.0 |
| strict_readiness | 1 |
| exit_timeout | 30 |
85 Views
0
Answers
8 months ago
8 months ago