Encountered this issue too. For k8s_glue_example.py I added in argument —overrides-yaml=<yaml file> which contains

spec:
containers:

• resources:
  limits:
  http://nvidia.com/gpu: 1

The pod is able to be allocated with gpu.