Ad1. yes, think this is kind of bug. Using _task to get pipeline input values is a little bit ugly
Good point, let;s fix it 🙂
new pipeline is built from scratch (all steps etc), but by clicking "NEW RUN" in GUI it just reuse existing pipeline. Is it correct?
Oh I think I understand what happens, the way the pipeline logic is built, is that the "DAG" is created the first time the code runs, then when you re-run the pipeline step it serializes the DAG from the Task/backend.
The initial thinking is that, well we want to sometime in the future allow you to easily edit the DAG in the UI, hence the behavior.
But, specifically here, we want the opposite.
As a temp hack you can add the following:
` print(pipe._task.get_parameters_as_dict())
clear the stored DAG
pipe._task.set_configuration_object(name=pipe._config_section, config_text="") `
Hi AgitatedDove14 ,
Ad1. yes, think this is kind of bug. Using _task to get pipeline input values is a little bit ugly 🙂
Ad2. I am not sure, I ask a question then:
When I run pipeline from script, new pipeline is built from scratch (all steps etc), but by clicking "NEW RUN" in GUI it just reuse existing pipeline. Is it correct?
Your question may get positive answer then and I can imagine explanation for that. I am just thinking if it working like that by design or not.
Hi HandsomeGiraffe70
First:# During pipeline initialisation pipeline_params is empty and we need to use default values. # When pipeline start the run, params are lunched again, and then pipeline_params can be used.
Hmm that should probably be fixed, maybe a function on the pipeline to deal with it ?
When I reduce tune_optime value to just 'recall'. Pipeline execution failed with msg:
ValueError: Node 'tune_et_for_Precision', base_task_id is empty
.
I would imagine that it is failing to find the requested Task?
specifically:project_name='Test/Research', task_name=f"tune_{model_name}_{item}",
obviously we should improve the error , but first could that be the case ?