Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone ! I'M Working With Pipelinecontroller. Is It Possible To Create A Pipeline With Optional Steps ? To Clarify I Add An Example Of A Pipeline In The Picture. For Example, The User Would Be Able To Modify The Value Of A Parameter To Execute The St

Hi everyone ! I'm working with PipelineController. Is it possible to create a pipeline with optional steps ? To clarify I add an example of a pipeline in the picture. For example, the user would be able to modify the value of a parameter to execute the steps from the red lines (step 1, 2.1 and 3) or yellow line (step 1 and 3) or green lines (step 1, 2.2 and 3)
image

  
  
Posted 2 months ago
Votes Newest

Answers 7


I do this a lot. pipeline params spawn K number of nodes, that collect just like you drew. No decorator being used here, just referencing tasks by id or name/project. I do not use continue on fail at all.

I do this with functions that have the contract ( f(pipe: PipelineController, **kwargs) -> PipelineController ) and a for-loop.

just be aware DAG creation slows down pretty quickly after a dozen or so such loops.

All the images below were made with the same pipeline (just evolved some names over time) + different parameter options. Including making the "collect results" step optional.
image
image
image

  
  
Posted 2 months ago

Hi @<1546303293918023680:profile|MiniatureRobin9> , if you use pipelines from decorator your can certainly set if statements to decide where/how to go

  
  
Posted 2 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36> , I'm using pipeline from tasks, am I able to do the same as pipeline from decorator ?

  
  
Posted 2 months ago

Okay, I found it : I put an if statement with the parameter and continue_on_fail=True in the steps, and it worked. But I had to specify the continue_on_fail in add_step to make it work.

  
  
Posted 2 months ago

Thank you for your help 😄

  
  
Posted 2 months ago

Yes, and even more 🙂

  
  
Posted 2 months ago

I suggest you see this example - None
And see how you can implement an if statement into this sample to basically create 'branches' in the pipeline 🙂

  
  
Posted 2 months ago