Unanswered
Hi All!
I Noticed When A Pipeline Fails, All Its Components Continue Running. Wouldn'T It Make More Sense For The Pipeline To Send An Abort Signal To All Tasks That Depend On The Pipeline?
I'M Using Clearml V1.1.3Rc0 And Clearml-Agent 1.1.0
The scheme is similar to the following:main_pipeline (PipelineDecorator.pipeline) | |----------------------------------| | | inference_orchestrator_1 inference_orchestrator_2 (PipelineDecorator.component, (PipelineDecorator.component, acting as a pipeline) acting as a pipeline) | | | | data_preprocessing_1 data_preprocessing_2 (PipelineDecorator.component) (PipelineDecorator.component) | | | | inference_1 inference_2 (PipelineDecorator.component) (PipelineDecorator.component) | | | | output_predictions_1 output_predictions_2
So if any step corresponding to 'inference_orchestrator_1' fails, then 'inference_orchestrator_2' keeps running. Is there any way to interrupt the main pipeline in case of failure of any of the inference pipelines?
171 Views
0
Answers
3 years ago
one year ago