But maybe only one step in the dag is flawed and I want to continue the rest of the pipeline as usual (despite the branch of the flawed task).
I am not sure what you mean by automatic stopping flows, could you give an example?
I aborted the task because of a bug on my side
Why? The task should have completed successfully, how is this aborting?
Early stopping by the HPO process, like hyper-band, e.g. this training model is going nowhere let's stop it.
SmarmySeaurchin8 it could be a switch, the problem is that when you have automatic stopping flows, they will abort a task, which is legitimate (e.g. should not considered failed)
How come you have aborted tasks in the pipeline ? If you want to abort the pipeline, you need to first abort the pipeline Task then the tasks themselves.
I think it should be treated as failed, I am truly not convinced as why aborting a task should be anything beside a user terminating an unwanted behavior of the task (be it bug, running with wrong config, task getting stuck etc..)
For example HPO, early stopping. It would mark the Task as aborted. Make sense ?
If this could be so, I'd be happy to have this as a feature, this really impacts my pipeline flow.
For example HPO, early stopping. It would mark the Task as aborted
Why? The task should have completed successfully, how is this aborting?
I aborted the task because of a bug on my side
🙂
Following this one, is treating abort as failed a must feature for the pipeline (in your case) or is it sort of a bug in your opinion ?
I think it should be treated as failed,
I'm not sure where I stand on default behavior, it it could easily be an argument for the pipeline controller