To clarify, there might be cases where we get helm chart /k8s manifests to deploy a inference services. A black box to us.

Users may need to deploy this service where needed to test out against other software components. This needs gpu resources which a queue system will allow them to queue up and eventually get this deployed instead of hard resource allocation to this purpose