Okay, I'll make sure we always qoute " , since it seems to work either way.
We will release an RC soon, with this fix.
Sounds good?

Also what is the docker vserion?

Okay, checking...

Hi OutrageousGrasshopper93
Are you working with venv or docker mode?
Also notice that is you need all gpus you can pass --gpus all

Are you using zsh by any chance?

yes

amazing and thanks! keep me posted

Hmm, let me check something

Docker version 19.03.7, build 7141c199a2 on Linux, btw

Ubuntu? which version?

Are you working with venv or docker mode?

sorry, important info! Docker mode

Also notice that is you need all gpus you can pass 

--gpus all

yes, i know, but i need to use 2 out of 4 for a queue

BTW:

Error response from daemon: cannot set both Count and DeviceIDs on device request.

Googling it points to a docker issue (which makes sense considering):
https://github.com/NVIDIA/nvidia-docker/issues/1026
What is the host OS?

OutrageousGrasshopper93 is "--gpus all" working ?

indeed, i managed to make a docker run command to work with the fix you mentioned ( docker run --gpus '"device=1,2"' nvidia/cuda:9.0-base nvidia-smi ) but trains-agent just appends to --gpus device= and there is no way to make the quoting like this

no, it's SUSE on a server, and bash