Hi @<1523701205467926528:profile|AgitatedDove14> , i got a strange behavior also with this RC. For pipeline itself and simple functions it works well as before. But for pipeline components I still get an error. Below are some examples based on the code I have shared initially (I have slightly modified it).
With the old clearml version 1.12.2.
If I remove @my_decorator from my_pipeline_component_with_wrapper pipeline runs successfully and produces following output:
--- main started
ClearML Task: ...
--- my_pipeline started
### my_pipeline
--- my_func started
### my_func from my_pipeline
--- my_func ended
Launching step [my_pipeline_component_no_wrapper]
ClearML results page: ...
### my_pipeline_component_no_wrapper foo
--- my_func started
### my_func from my_pipeline_component_no_wrapper
--- my_func ended
Launching step [my_pipeline_component_with_wrapper]
--- my_pipeline ended
ClearML results page: ...
### my_pipeline_component_with_wrapper foo
--- main ended
When I add @my_decorator to my_pipeline_component_with_wrapper pipeline fails:
--- main started
--- internal_decorator started
ClearML Task: created ...
### my_pipeline
--- my_func started
### my_func from my_pipeline
--- my_func ended
Launching step [my_pipeline_component_no_wrapper]
--- my_pipeline_component_with_wrapper started
ClearML results page: ...
### my_pipeline_component_no_wrapper foo
--- my_func started
### my_func from my_pipeline_component_no_wrapper
--- my_func ended
Launching step [my_pipeline_component_with_wrapper]
--- my_pipeline_component_with_wrapper ended
Traceback (most recent call last):
File "C:\Users\XXXXX~1\AppData\Local\Temp\tmptvtlgewe.py", line 11, in <module>
@my_decorator
^^^^^^^^^^^^
NameError: name 'my_decorator' is not defined
Setting pipeline controller Task as failed (due to failed steps) !
Traceback (most recent call last): ...
It also seems to play a role if I put @my_decorator above or under @PipelineDecorator.
- The first example had @my_decorator below @PipelineDecorator for my_pipeline.
- The second example was for @my_decorator placed above @PipelineDecorator for both piepline and pipeline component.As you might see for the pipeline instead of "--- my_pipeline started" i got "--- internal_decorator started"
If I place @my_decorator under @PipelineDecorator for both pipeline and components - following output is produced and process freezes:
--- main started
ClearML Task: created ...
--- my_pipeline started
### my_pipeline
--- my_func started
### my_func from my_pipeline
--- my_func ended
Launching step [my_pipeline_component_no_wrapper]
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "...\decorators.py", line 7, in wrapper
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "...\pipelines.py", line 23, in main
result = my_pipeline()
^^^^^^^^^^^^^
File "...\.venv\Lib\site-packages\clearml\automation\controller.py", line 4339, in internal_decorator
pipeline_result = func(**pipeline_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "....\decorators.py", line 7, in wrapper
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "...\pipelines.py", line 14, in my_pipeline
c = my_pipeline_component_with_wrapper(b) # <----- it fails here
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "....\.venv\Lib\site-packages\clearml\automation\controller.py", line 3906, in wrapper
kwargs[inspect_func.args[i]] = v
~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
PS ... ClearML results page: ....
### my_pipeline_component_no_wrapper foo
--- my_func started
### my_func from my_pipeline_component_no_wrapper
--- my_func ended
With the new version (1.13.3rc1), when I place @my_decorator under pipeline decorator I get exactly the same error as in the second example. For the case when custom decorators are placed above pipeline decorator following output is produced (and it still fails):
--- main started
--- internal_decorator started
ClearML Task: ...
### my_pipeline
--- my_func started
### my_func from my_pipeline
--- my_func ended
Launching step [my_pipeline_component_no_wrapper]
--- my_pipeline_component_with_wrapper started
ClearML results page: ...
### my_pipeline_component_no_wrapper foo
--- my_func started
### my_func from my_pipeline_component_no_wrapper
--- my_func ended
Launching step [my_pipeline_component_with_wrapper]
--- my_pipeline_component_with_wrapper ended
ClearML results page: ...
--- my_pipeline_component_with_wrapper started
ClearML pipeline page: ....
Launching step [my_pipeline_component_with_wrapper]
--- my_pipeline_component_with_wrapper ended
Traceback (most recent call last):
File "C:\Users\XXXXX~1\AppData\Local\Temp\tmpag35j399.py", line 31, in <module>
@my_decorator
^^^^^^^^^^^^
NameError: name 'my_decorator' is not defined
So it looks like behavior changed, but still results in an error. Also it looks like the order (and number) of function calls is a bit chaotic 😢