Essentially, I think the key thing here is we want to be able to build the entire Pipeline including any updates to existing pipeline steps and the addition of new steps without having to hard-code any Task ID’s and to be able to get the pipeline’s Task ID back at the end.

Oh if this is he case then basically you CI/CD code will be something like:

@PipelineDecorator.component(return_values=['data_frame'], cache=True, task_type=TaskTypes.data_processing)
def step_one(pickle_data_url: str, extra: int = 43):
    print('step_one')

@PipelineDecorator.pipeline(pipeline_execution_queue=None, name='custom pipeline logic', project='examples', version='0.0.5')
def executing_pipeline(pickle_url, mock_parameter='mock'):
    step_one(pickle_url)

PipelineDecorator.set_default_execution_queue('default')
executing_pipeline(pickle_url='

')

That's it. The components are executed remotely but the pipeline logic itself is executed on the github actions node (i.e. this is just logic, no actual compute / data)
Notice the key is the pipeline_execution_queue=None, argument that basically tells the pipeline logic it should run locally (components are still running remotely)
wdyt?