Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
<no title>

  • Is there a reference implmentation for a task in a pipeline that awaits user input?
  
  
Posted 3 years ago
Votes Newest

Answers 12


thanks so much for taking the time by the way!

  
  
Posted 3 years ago

So “wait” is a better metaphore for me

So I would do something like (I might have a few typos but that's the gist):


def post_execute_callback_example(a_pipeline, a_node):
    # type (PipelineController, PipelineController.Node) -> None
    print('Completed Task id={}'.format(a_node.executed))
    # wait until model is tagged, then pass it as argument
    while True:
        found = Moodel.query_models(...) # model filter here, inlucing tag and project
        if found:
            break
        print('waiting for model taggging')
        sleep(60)
    # find the actual next step node and pipe the model model ID
    next_step_node= a_pipeline.get_pipeline_dag()['next_step_name_here']
    next_step_node.parameters['Args/model_id'] = found[0].id
    return

pipe.add_step(name='stage_process', parents=['stage_data', ],
              base_task_project='examples', base_task_name='pipeline step 2 process dataset',
              parameter_override={'General/dataset_url': '${stage_data.artifacts.dataset.url}',
                                  'General/test_size': 0.25},
              post_execute_callback=post_execute_callback_example
              )

WDYT?

  
  
Posted 3 years ago

Hi @<1523704198338711552:profile|RoughTiger69> ,

there is a pipeline example that gets the input from previous tasks in the pipeline, can this help?

None

  
  
Posted 3 years ago

yes and no.
This is a pseudo flow:
Data download -> pre-processing -> model training (e.g. HPT) - > model evaluation (per variant) -> model comparison dashboard -> human selects the best model using a heuristic and the status of the weather -> model packaging -> inference tests etc.

I could divide it into two pipelines:
Data download --> dashboard
Packaging --> …

Where packaging takes a parameter which is the human selected ID of the model.
However, this way, I lose the context of the entire flow.
I also don’t have a way to “push” the human to make a decision or reflect it back to them.

So I am looking for something that is similar to manual QA approval step in CI/CD.

  
  
Posted 3 years ago

is there a way to visualize the pipeline such that this step is “stuck” in executing?

Yes there is, the pipelline plot (see plots section on the Pipeline Task, will show the current state of the pipeline.
But I have a feeling you have something else in mind?
Maybe add Tag on the pipeline Task itself (then remove it when it continues) ?
I'm assuming you need something that is quite prominent in the UI, so someone knows ?
(BTW I would think of integrating it with the slack monitor, to push a p=message to users that the pipeline is waiting for human interaction), wdyt ?

  
  
Posted 3 years ago

yeah, its a tradeoff that is dependent on parameters that lie outside the realm of human comprehension.
Let’s call if voodoo.
Yes, the manual selection can be done via tagging a model.
The main thing is that I want the selection to be part of the overall flow.
I want the task of human tagging a model to be “just another step in the pipeline”

  
  
Posted 3 years ago

sounds workable, will start with that

  
  
Posted 3 years ago

Hi @<1523704198338711552:profile|RoughTiger69>
From this scenario can we assume the "selection" will be tagging the model manually?
Also, how would an human operator decide on the best model, that is what is the input to base the decision on?

  
  
Posted 3 years ago

that’s the thing. I want to it to appear like one long pipeline, vs. trigger a new set of steps after the approval. So “wait” is a better metaphore for me

  
  
Posted 3 years ago

could work! is there a way to visualize the pipeline such that this step is “stuck” in executing?

  
  
Posted 3 years ago

Awesome! let me know how/if it works 🙂

  
  
Posted 3 years ago

I want the task of human tagging a model to be “just another step in the pipeline”

That makes total sense.
Quick question, would you prefer the pipeline controller to "wait" for the tagging and then continue, or would it make more sense to create a trigger on the tagging ?

  
  
Posted 3 years ago
516 Views
12 Answers
3 years ago
8 months ago
Tags