Not sure about that, I think you guys solved it with your PipelineController implementation. I would need to test it before giving any feedback 🙂

I think you solved it 🙂
How do you imagine a slicker interface ?

And I do that each time I want to create a subtask. This way I am sure to retrieve the task if it already exists

Basically what I did is:
` if task_name is not None:
project_name = parent_task.get_project_name()
task = Task.get_task(project_name, task_name)
if task is not None:
return task

Otherwise here I create the Task `

it should return the task regardless if it is complete or not

the only problem with it is that it will start the task even if the task is completed

What is the criteria ?

continue_last_task is almost what I want, the only problem with it is that it will start the task even if the task is completed

JitteryCoyote63 Great to hear 🙂
BTW:

Would it be possible to extend 

Task.init

  with a 

force_reuse

 that would enforce reusing these tasks

You can pass continue_last_task=True I think it should be equivalent to what you suggest

AgitatedDove14 This looks awesome! Unfortunately this would require a lot of changes in my current code, for that project I found a workaround 🙂 But I will surely use it for the next pipelines I will build!

All the 3 steps can be found here:
https://github.com/allegroai/trains/tree/master/examples/pipeline

Hi JitteryCoyote63
The new pipeline is almost ready for release (0.16.2),
It actually contains this exact scenario support.
Check out the example, and let me know if it fits what you are looking for:
https://github.com/allegroai/trains/blob/master/examples/pipeline/pipeline_controller.py