Also, in the link above there is the warning

Do not enqueue training or inference tasks into the services queue. They will put an unnecessary load on the server.

I am not using the dedicated

services

queue on the server but I am doing training and inference in the pipeline component.

Steps of a pipeline should have dedicated queues with relevant resources to them

Really depends on how you want to set up your pipeline. I suggest going over the documentation and watching the youtube videos for a better understanding.

Ah ok, so if I have two steps, this means 3 queues with 3 agents, one for the controller and one each dedicated to the steps, just to confirm?

I assigned both the pipeline controller and the component to this worker. Do I rather need to create two agents, one in services mode for the controller and then another one (not in services mode) for the component (which does training and predictions)? But, this seems to defeat the point of being able to run multiple tasks in services mode...

Yes. Again, the services mode is for special 'system' services if you will. The controller can run on the services agent (although not necessary since you can run the pipeline controller on any queue you'd like) while the steps themselves run on different queues

Thanks, I'll have a look at the youtube videos. I've been going over the documentation a lot and haven't found much about actually running and deploying pipelines, and a lot of details seem to be missing.