AgitatedDove14 - on a similar note, using this is it possible to add to requirements of task with task_overrides?
pipeline, can I control the tags that the tasks a pipeline creates?
add_pipeline_tags
adds tags from pipeline to the tasks I suppose? But I also need to clear existing tags in those created tasks
add_pipeline_tags will add the unique ID of the pipeline execution, if you want to add specific tags you can use the task_overrides and provide:pipe.add_step(..., task_overrides={'tags': ['my', 'tags']})
using this is it possible to add to requirements of task with task_overrides?
Correct, but you will be replacing (not adding) requirements