if I build a custom image, do I have to host it on dockerhub for it to run on the agent?

You dont need to host it, but in this case the machine running the agent should have the image (you can verify on the machine with docker images ).

If not how do I make the agent aware of my custom image?

Once the image is the base docker image for this task, and the image was verify on the agent’s machine, the agent should be able to use it