Do notice this will only work for pipelines from Tasks, is this a good fit for you?

The issue with this is that we would then presumably have to run/“build” each of the Tasks (pipeline steps) separately to put them on the ClearML server and then get their Task ID’s in order to even write the code for the Pipeline, which increases the complexity of any automated CI/CD flow. Correct me if I’m wrong.

Essentially, I think the key thing here is we want to be able to build the entire Pipeline including any updates to existing pipeline steps and the addition of new steps without having to hard-code any Task ID’s and to be able to get the pipeline’s Task ID back at the end.
The functions and decorators pipelines are great because they handle the Task ID’s of the individual steps implicitly.

pipe = Pipeline(...)
pipe.add_step(...)
task_id = pipe.build()
print(f"pipeline task ID = {task_id}")

Having a pipe.build() that creates a “draft” version of the pipeline on ClearML and returns the task id would be perfect, and it would be even more perfect if we could do this from functions or decorators, as individually “building” the steps of the Pipeline will make this much more complex. If it has to be done, it has to be done, but we would also need some way of doing the exact same thing (programmatically “build” and get the Task ID) with each of the Tasks in the Pipeline.