I don't know why it didn't detect it in first place

thanks, I will do that

I have 11.0 installed but on another machine with 11.0 installed as well, trains downloads torch for cuda 10.1, I guess this is because no wheel exists for torch==1.3.1 and cuda 11.0

It is configured as CPU (i.e. no CUDA)

agent.cuda_version = 0 agent.cudnn_version = 0

You're welcome 🙂

python3 -m trains_agent --config-file "~/trains.conf" daemon --queue default --log-level DEBUG --detached --gpus 1 > ~/trains-agent.startup.log 2>&1

what do you see in the console when you start the trains-agent , it should detect the cuda version

okay, now it should work 🙂

What you actually specified is torch the @ is kind of pip remark, pip will not actually parse it 🙂
use only the link https://download.pytorch.org/whl/cu100/torch-1.3.1%2Bcu100-cp36-cp36m-linux_x86_64.whl

Hi JitteryCoyote63
What do you have in the agent.cuda_version ?
(you can see it printed at the beginning of the log)

agent.cuda_version = 110 agent.cudnn_version = 0

yes, I use --gpus parameter

AgitatedDove14 one last question: how can I enforce a specific wheel to be installed?

I just started one and it wrote:
...

That depends on what you have installed 🙂

(since you are using venv mode, if the cuda is not detected at startup time, it will not install the GPU version, as it has no CUDA support)

This is also set in the command line.
--cpu-only or maybe without any --gpus flag at all

You can always force it with environment variable CUDA_VERSION=10.1

cuda 10.1, I guess this is because no wheel exists for torch==1.3.1 and cuda 11.0

Correct

how can I enforce a specific wheel to be installed?

You mean like specific CUDA wheel ?
you can simple put the http link to the wheel in the "installed packages", it should work

OK but nowhere I specified that, I just checked my trains.conf file

This is why it thinks it's CPU 🙂

I specified a torch @ https://download.pytorch.org/whl/cu100/torch-1.3.1%2Bcu100-cp36-cp36m-linux_x86_64.whl and it didn't detect the link, it tried to install latest version: 1.6.0

btw shoulnd't it be CUDA_VERSION=11.0 ?

ho, that might be it then, thanks!