Unanswered
<no title>
So “wait” is a better metaphore for me
So I would do something like (I might have a few typos but that's the gist):
def post_execute_callback_example(a_pipeline, a_node):
# type (PipelineController, PipelineController.Node) -> None
print('Completed Task id={}'.format(a_node.executed))
# wait until model is tagged, then pass it as argument
while True:
found = Moodel.query_models(...) # model filter here, inlucing tag and project
if found:
break
print('waiting for model taggging')
sleep(60)
# find the actual next step node and pipe the model model ID
next_step_node= a_pipeline.get_pipeline_dag()['next_step_name_here']
next_step_node.parameters['Args/model_id'] = found[0].id
return
pipe.add_step(name='stage_process', parents=['stage_data', ],
base_task_project='examples', base_task_name='pipeline step 2 process dataset',
parameter_override={'General/dataset_url': '${stage_data.artifacts.dataset.url}',
'General/test_size': 0.25},
post_execute_callback=post_execute_callback_example
)
WDYT?
89 Views
0
Answers
3 years ago
8 months ago