Hi VivaciousPenguin66
Seems like a CUDA/CUDNN issue.
You argent is configured to work in venvmode, which mean it will pull the correct pytorch version based on the detected CUDA driver support. Speicifally you can see in the log "agent.cuda_version = 111" which means CUDA 11.1 and from the log it found the correct pytorch version:Torch CUDA 111 download page found Found PyTorch version torch==1.8.1 matching CUDA version 111 Found PyTorch version torchvision==0.9.1 matching CUDA version 111 Collecting torch==1.8.1+cu111 File was already downloaded /home/edmorris/.clearml/pip-download-cache/cu111/torch-1.8.1+cu111-cp38-cp38-linux_x86_64.whl ...
The error itself seems like pytorch/cuda compatibility issue, not directly connected with clearml
, no?