Nice, in the meantime as a workaround I will implement a temporary parsing code at the beginning of step functions
... transformed to 'str' when passed to a function decorated with
PipelineDecorator.component
at the time of calling it in the pipeline itself. Again, is this something intentional?
Are you sure about that? Notice the example code specifies, int as well...
GiganticTurtle0 I'm not sure I follow, what do you mean by indexing the arguments? Can you post a short usage example ?
Sure, I will post a mock example in a while
I have also tried with type hints and it still broadcasts to string. Very weird...
Exactly, when 'extra' has a default value (in this case, 43), the argument preserves its original type. However, when 'extra' is a positional argument then it is transformed to 'str'
However, when 'extra' is a positional argument then it is transformed to 'str'
Hmm... okay let me check something
BTW, I would like to mention another problem related to this I have encountered. It seems that arguments of type 'int', 'float' or 'list' (maybe also happens with other types) are transformed to 'str' when passed to a function decorated with PipelineDecorator.component
at the time of calling it in the pipeline itself. Again, is this something intentional?
GiganticTurtle0 hi!
Do you have an example of your use case?
Okay the type is inferred from the default value of the function step itself, that means that both:data_frame = step_one(pickle_url, extra=1337)
anddata_frame = step_one(pickle_url, 1337)
Will pass extra as int
.
That said if the default value of the argument is missing, it will revert to str
In order to use the type hints as casting hint, we actually need to improve the task.connect
to support the type casting (they are stored)
I have also tried with type hints and it still broadcasts to string. Very weird...
Type hints are ignored, it's the actual value you pass that is important:
` @PipelineDecorator.component(return_values=['data_frame'], cache=True, task_type=TaskTypes.data_processing)
def step_one(pickle_data_url: str, extra: int = 43):
...
@PipelineDecorator.pipeline(name='custom pipeline logic', project='examples', version='0.0.5')
def executing_pipeline(pickle_url, mock_parameter='mock'):
data_frame = step_one(pickle_url, extra=1337) implies
extra ` is an integer Because this is the value that was passed by the pipeline controller when calling the function
make sense ?
GiganticTurtle0 quick update, a fix will be pushed, so that casting is based on the Actual value passed not even type hints 🙂
(this is only in case there is no default value, otherwise the default value type is used for casting)