There is an annoying bug in Docker mode which I have noticed several months ago but put up with it for some reason, despite I kept running into it all the time.

Sometimes agents don't pull layers for updated Docker images if the image with the same name and tag (i.e. previous version) is already present there. For instance, this very often happens when I add new pip packages to an image. This leads to ImportError s when I run the task. If I ssh to an agent and docker image rm the image manually, then tasks pull the whole image with correct packages.