Unanswered
What Happens To File That Are Downloaded To A Remote_Execution Via Storagemanager? Are They Removed At The End Of The Run, Or Does It Continuously Increases Disk Space?
Honestly, this is all related to issue #340. The only reason we have this to begin with is because we need one separate "initializer" task that downloads the remote cache and prepares the agent environment for execution (downloading the configuration files, etc).
Otherwise it fits perfectly with pipelines, but we're not there yet.
In the local execution we don't have this initializer task, so we use Task.init()
before starting to work on a model, and task.close()
when we're done.
I'd suggest some task.detach()
method for remote execution maybe? We still find use-cases for this initializer task (it holds the original user and as such we can easily filter and report it to Slack, etc).
154 Views
0
Answers
2 years ago
one year ago
Tags