Hi everyone,
I’m encountering a CPU bottleneck while performing inference with ClearML Serving and am hoping to get some assistance.
Setup: I have successfully deployed a ClearML Server and configured ClearML Serving following the instructions provided here: ClearML Serving Setup . I’m specifically using the docker-compose-triton.yml file, as I’m working with ONNX models.
Issue: During inference on ClearML Serving, I’ve noticed that only a single CPU core is being utilized, while the remaining cores remain idle. This causes the inference to time out each time. Is there a way to distribute the workload across multiple CPU cores to improve performance?
Thanks in advance for any help or suggestions!