I got it running, and now I know some more tricks. Which is good. Part of it was me facing this really weird PyTorch + setuptools issue: https://github.com/pytorch/pytorch/issues/69894

yes, you could also use the container’s SETUP SHELL SCRIPT and run command to install your python version (e.g.
sudo apt install python3.8for example)

TimelyPenguin76 I ran the agent with Docker as
$clearml-agent daemon --detached --gpus 0 --queue idx_gandalf_titan-rtx --docker nvidia/cuda:11.2.2-cudnn8-runtime-ubuntu18.04However, that nvidia image is Python 3.6, so when I run anything it's falling back to python 3.6. Thus, I thought maybe all I would need to do is clone the job and change the Docker image to an image with Python 3.8 and the right Pytorch, etc. Is my thinking right? I'm finding the right image to use now.

Hi GleamingGrasshopper63 , can you share how you are running the clearml agent (with venv, docker, if docker, your image?)?
For updating the agent configuration after its started to run, you’ll need to restart the agent 🙂