Unanswered
How Can I Send A Composed Chunk Of Code For Remote Execution
Ah, I think I understand. To execute a pipeline remotely you need to use None pipe.start() not task.execute_remotely . Do note that you can run tasks remotely without exiting the current process/closing the notebook, (see here the exit_process argument None ) but you won't be able to return any values from this task.
Given my understanding, I would actually recommend you the following:
- For each remote run, create a new
taskfrom a function like this: None - Enqueue the created
task: None this will not exit the current process - To be able to retrieve results from the remotely executed
task, you will have to manually save artifacts in the function that the remotetaskis executing. Because you have thetaskobject in your notebook process, you can then retrieve these artifacts upon completion None
249 Views
0
Answers
one year ago
one year ago