could one also limit the number of CPU cores available?

If you are running in docker mode you can add:
--cpus=<value>see ref here: https://docs.docker.com/config/containers/resource_constraints/

Just add it to extra_docker_arguments :
https://github.com/allegroai/clearml-agent/blob/2cb452b1c21191f17635bcb6222fa8bfd82afe29/docs/clearml.conf#L142

Thought it might be via docker, thanks!

Is there some default Docker image you ship with ClearML that you'd recommend, or can/should we use our own? 🙂

You mean for running a worker? (I think plain vanilla python / ubuntu works)
The only change would be pip install clearml / clearml-agent ...