AgitatedDove14

So nope, this doesn't solve my case, I'll explain the full use case from the beginning.

I have a pipeline controller task, which launches 30 tasks. Semantically there are 10 applications, and I run 3 tasks for each (those 3 are sequential, so in the UI it looks like 10 lines of 3 tasks).

In one of those 3 tasks that run for every app, I save a dataframe under the name "my_dataframe".
What I want to achieve is once all tasks are over, to collect all those "my_dataframe" artifacts (10 in number), extract a single line from each, and concatenate them to a new dataframe. I want to register this new dataframe to the pipeline task. It's kind of a summary of the most important detail in the process.

Now, first of all I'm not sure how to get to all those children tasks, but that might be solveable by using monitor_artifact - but then I wonder if they are all called the same how will I be able to extract the single row I need from them?

Then, assuming we solved that, I want to concatenate those rows and save them as a new dataframe to the pipeline task...

Hope this makes it clearer and you can tell me if this is possible, and if so, how to do it