Unanswered
Two Questions About Datasets:
Question 1: Are Parallel Writes To A Dataset With The Same Version Possible? Is The Way To Go, To Have A Task, Which Creates A Dataset Object, Which In Turn Is Passed As Artifact To The Subsequent Ingestion Tasks? After The P
to question 1:
passing Dataset artifacts between tasks seems to be not possible, getting the following error message:
TypeError: cannot pickle '_thread.lock' object.
So i guess its not possible to upload files from different tasks in parallel to the dataset, before finalizing it.
125 Views
0
Answers
10 months ago
10 months ago