AgitatedDove14 is that the expect behavior for Pipelines?
Hi IrritableGiraffe81
PipelineDecorator.debug_pipeline() runs everything as regular python functions, but "PipelineDecorator.run_locally()" is actually sumulating all the steps on the same local machine (so that it is easier to debug the "real" pipeline running on multiple machines)
What I think is happening is that the casting of the arguments passed to the component fail.
Basically the type hints are currently ignored (we are working on using them for casting in the next version)
but right now, the casting is done based on default values. This is why you fix worked 🙂
Found the issue.
For some reason, all parameters on the main functions are passed as strings.
So I have these parameters:
@PipelineDecorator.pipeline(name='Build Embeddings', project='kgraph', version='1.3') def main(tk_list=[], ngram_size=2): ...
The ngram_size variable is a int when using PipelineDecorator.debug_pipeline()
and it is a string when I used PipelineDecorator.run_locally()
I’ve add Python type hints and it fixed the issues:def main(tk_list:list = [], ngram_size:int = 2):