Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, Can Anyone Help Me? I'M Trying To Create A

Hi,
Can anyone help me? I'm trying to create a pipeline of pipelines instead of pipeline of tasks?

  
  
Posted one year ago
Votes Newest

Answers 12


I'm using the PipelineController to ctreate a pipeline.
now I want to add 2 other pipelines to this main-pipeline using add_step (so instead of task-id I'm giving it a pipeline-id.
but it doesn't work and always throws error.

If I'm doing the same thing with task ids all works.. so I guess there is another correct way to do a pipeline of pipelines

  
  
Posted one year ago

@<1559711623147425792:profile|PlainPelican41> , I think you should switch to using pipelines from decorators for easier management of your tasks/code. Creating pipelines from pipelines according to your scenario (difficulty managing code) would only lead to more difficulty tracking the code executions - How would you show pipelines of pipelines results? How many depths would be enabled? How should data aggregation work? This introduces many problems without any benefit.

I think restructuring the way you manage your pipelines might be the way 🙂

  
  
Posted one year ago

currently yes.
but as I said I want to create a pipeline of pipelines

  
  
Posted one year ago

earlier you've said I can managed to do so with some hacks. do u have an example or can u explain what I need to do 🙂 ?
maybe some hacks will be ok for now too

  
  
Posted one year ago

I'm not sure pipeline of pipelines would work out of the box without some hacks. What is your use case though, why do you need a pipeline of pipelines? In the end it would abstract into a single large pipeline anyways

  
  
Posted one year ago

Hi @<1559711623147425792:profile|PlainPelican41> , what exactly are you trying to do and what is your use case?

  
  
Posted one year ago

image

  
  
Posted one year ago

Are you using pipelines from tasks?

  
  
Posted one year ago

let's say I have pipeline_1 which has 10 tasks in it.
and I also have pipeline_2 with has 10 tasks in it.
I want a new_pipeline that will run pipeline_1 and after that will run pipeline_2.

If I cannot do pipeline of pipelines, it means that I need to create a pipeline containing 20 tasks using add_step !
and In the future if I would like to add pipeline_3 to all of this, I'll need again to use add_step to all the 20 task + the ones of pipeline_3.

I makes all the process very uncomfortable as we scales to larger automations and tasks. So I need a way to make it simple and more readable using a pipeline of pipelines

  
  
Posted one year ago

@<1559711623147425792:profile|PlainPelican41> what is the exact use-case? Why isn't a single pipeline enough? Logically a pipeline of pipelines is equivalent to a single pipeline...

  
  
Posted one year ago

I'm thinking maybe you can do it by chaining tasks somehow, but I think that is not the correct way (Never tried it myself, as I said, in the end it all abstracts to a single pipeline).

Maybe check out pipelines with functions or even better decorators, this might be an easier solution for you and you can create very complex pipelines with it. Think about using loops and if statements to create steps and progress your pipeline

  
  
Posted one year ago

exactly because of the reason I described above:
None

  
  
Posted one year ago
657 Views
12 Answers
one year ago
one year ago
Tags
Similar posts