Unanswered
How Can I Send A Composed Chunk Of Code For Remote Execution
Ah, I think I understand. To execute a pipeline remotely you need to use None pipe.start()
not task.execute_remotely
. Do note that you can run tasks remotely without exiting the current process/closing the notebook, (see here the exit_process
argument None ) but you won't be able to return any values from this task.
Given my understanding, I would actually recommend you the following:
- For each remote run, create a new
task
from a function like this: None - Enqueue the created
task
: None this will not exit the current process - To be able to retrieve results from the remotely executed
task
, you will have to manually save artifacts in the function that the remotetask
is executing. Because you have thetask
object in your notebook process, you can then retrieve these artifacts upon completion None
110 Views
0
Answers
10 months ago
10 months ago