Hi, I'm using the aws autoscaler to spin instances. I'd like to use the clearml agent on the created instances with docker containers. However even if I set default_docker_image in the parameters on the UI to nvidia/cuda:11.1.1-runtime-ubuntu20.04 the task is still using the default nvidia/cuda:10.1-runtime-ubuntu18.04. This is weird also because when i ssh into the created istance the clearml-agent is running withpython -m clearml_agent --config-file /root/clearml.conf daemon --queue ml_pipeline --docker nvidia/cuda:11.1.1-runtime-ubuntu20.04 . Should I change some env variable in the ClearmlServer?
Hey LovelyHamster1 ,
Any chance the task you are trying to run has a base docker defined in it?
To check, go to the experiment's page and then to EXECUTION > AGENT CONFIGURATION > BASE DOCKER IMAGE
If it's set to any value, clearing it would solve your problem.
Yes it was set to nvidia/cuda:10.1-runtime-ubuntu18.04... ok I'll try again and see if that was the problem, thank you
