I would like to bypass this behavior because my code has a need for a specific version of PyTorch.
DilapidatedCow43 you will get exactly the pytorch version you need, but complied to the CUDA version that is installed (pytorch people actually maintain multiple versions based on different cuda versions)
you can try to update the clearml.conf it has agent.cuda_version