Unanswered
Question About Pipeline And Long-Waiting Tasks:
Say I Want To Generate A Dataset.
The Workflow I Have Requires
RoughTiger69 I think this could work, a pseudo example:
` @PipelineDecorator.component(...)
def the_last_step_before_external_stuff():
print("doing some stuff")
@PipelineDecorator.pipeline()
def logic():
the_last_step_before_external_stuff()
if not check_if_data_was_ingested_to_the_system:
print("aborting ourselves")
Task.current_task().abort()
# we will not get here, the agent will make sure we are stopped
sleep(60)
# better safe than sorry
exit(0) `wdyt? (the same logic can be implemented with pipeline from Tasks, and a callback function that essentially does the same)
When this pipeline is executed, it will not run "the_last_step_before_external_stuff()" again, and will just get to the if statement
179 Views
0
Answers
2 years ago
one year ago
Tags