Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Have Another Pipeline-Related Question. In A Pipeline Controller Task, I Would Like To Add Several Steps That Are Based On The Same Base Task (I'M Passing The

I have another pipeline-related question. In a pipeline controller Task, I would like to add several steps that are based on the same base Task (I'm passing the base_task_ID ), and run each with a different configuration (using parameter_override ). When I run the pipeline, the base Task is cloned several times as intended, but each cloned Task has the same configuration, despite passing a different dictionary to parameter_override for each clone. I'm not sure exactly why this is happening. Is this the intended behavior? If so, how can I modify my pipeline for my usecase (cloning one base task several times with different configs)? Thank you!

  
  
Posted 2 years ago
Votes Newest

Answers 5


Here's the example: Even though I'm passing different parameters to the two clones, they will end up configured with the same (the second) parameter set.
` project_name = 'pipeline_test'

task = Task.init(project_name=project_name,
task_name='pipeline_main',
task_type=Task.TaskTypes.controller,
reuse_last_task_id=False)

pipe = PipelineController(default_execution_queue='default_queue',
add_pipeline_tags=False)

#%%
task_IDs = {
'task_1': '',
'task_2': ''
}

#%%
task_1_args = {'param1', 1}
task_2_args_clone_1 = {'param1', 1}
task_2_args_clone_2 = {'param1', 2}

first task

pipe.add_step(name='task_1',
base_task_project=project_name,
base_task_id=task_IDs['task_1'],
parameter_override=task_1_args)

second tasks

pipe.add_step(name='task_2_clone_1',
parents=['task_1'],
base_task_project=project_name,
base_task_id=task_IDs['task_2'],
parameter_override=task_2_args_clone_1)

pipe.add_step(name='task_2_clone_2',
parents=['task_1'],
base_task_project=project_name,
base_task_id=task_IDs['task_2'],
parameter_override=task_2_args_clone_2) `

  
  
Posted 2 years ago

The section name needs to be added in both the base task as well as the pipeline task for it to work. Since the parameters also show up in the "General" section in the web interface when the parameters are connected only with their name (without section name), I didn't think that this could matter. Thank you for your help!

  
  
Posted 2 years ago

Thank you, this fixed the issue!

  
  
Posted 2 years ago

Hi SucculentBeetle7
The parameters passed to add_step need to contain the section name (maybe we should warn if it is not there, I'll see if we can add it).
So maybe something like:
{'Args/param1', 1}Or
{'General/param1', 1}Can you verify it solves the issue?

  
  
Posted 2 years ago

No worries, and I will make sure we output a warning if section names are not used 🙂

  
  
Posted 2 years ago