Unanswered
How Can I Send A Composed Chunk Of Code For Remote Execution
Hey @<1523701083040387072:profile|UnevenDolphin73> what you're building here sounds like a useful tool. Let me understand what you're trying to achieve here, please correct me if I'm wrong:
- You want to create a set of
Step
classes with which you can define pipelines, that will be executed either locally or remotely. - The pipeline execution is triggered from a notebook.
- The
steps
are predefined transformations, the user normally won't have to create their own steps
Did I get all of this right?
As for your two questions
- the pipeline controller expects either functions (
.add_function_step
or via@PipelineDecorator.component
) or already executed tasks (.add_step
), not classes. - You can ofc edit the steps, but by running the pipeline you will recreate it. Maybe it will help if you define what does "recreating a pipeline" mean to you and why is it a problem
116 Views
0
Answers
11 months ago
11 months ago