Hi @<1523704757024198656:profile|MysteriousWalrus11>
in the pipeline quickly between pipeline.add_step() functions?
You mean you want to get access to the parent Task ids and query them directly ?
I think the easiest way is to pass it as one of the parameters
(you can get to the pipeline Task itself from the running component, then get the dag, but these are internal functions, maybe we should make them external for easier querying ?)
pipe.add_step(
name="stage_process",
parents=["stage_data"],
base_task_project="examples",
base_task_name="Pipeline step 2 process dataset",
)
pipe.add_step(
name="stage_train",
parents=["stage_process"],
base_task_project="examples",
base_task_name="Pipeline step 3 train model",
parameter_override={"General/dataset_task_id": "${stage_process.id}"},
)
Notice the ${stage_process.id}
is the parent Task ID.