btw, when running pipelines, a common pattern is that two consequential components end up on the same node. The current implementation will upload the result of the first component, and then the first thing the next component will do is download it. I assume get_local_copy method is used and the output is stored to local cache. Wouldn’t it be more performant for the first component to store its result to the local cache along uploading it to file server? In that way, the next component if r...

Thanks for the response!
I understand I can change the docker image for a component in the pipeline, but for the https://github.com/allegroai/clearml/blob/90f30e8d9a5ca9a1afa6b2e5ffccb96b0afe9c78/examples/pipeline/pipeline_from_decorator.py#L77 it isn’t possible. I see that you can just change the queue it runs on, not the docker image, nor its params, nor the requirements. thanks for this! I’ll try it!

Thanks for the extensive response! As the solution seems a bit hackish, I’d definitely prefer the ability to set a docker image/docker args/requirements config for the pipeline controller too