AgitatedDove14 thanks, good idea.

My main issue with this approach is that it breaks the workflow into “a-sync” set of tasks:

One task sends a list of images for labeling and terminates an external webhook calls http://clear.ml and creates a dataset from the labels returned from the labeling task a trigger wakes up the label post processing/splitting logic.
It will be hard to understand where things are standing from looking at the UI.

I was wondering if the “waiting” operator can actually be a part of the pipeline.
This way it will look more clear what is the workflow we are executing.

because the wait for the external labeling is very long, I am not sure that long polling / sleeping inside the task is a good idea.

I was wondering if I could:
programmatically “abort” the pipeline. externally tell the pipeline to “resume” from the point it was aborted (e.g. specific task name).
WDYT?