Two Questions About Datasets: Question 1: Are Parallel Writes To A Dataset With The Same Version Possible? Is The Way To Go, To Have A Task, Which Creates A Dataset Object, Which In Turn Is Passed As Artifact To The Subsequent Ingestion Tasks? After The P

Unanswered

to question 1:
passing Dataset artifacts between tasks seems to be not possible, getting the following error message:

TypeError: cannot pickle '_thread.lock' object.

So i guess its not possible to upload files from different tasks in parallel to the dataset, before finalizing it.

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SaltySpider22
				
					0
					 × 1

270 Views

0 Answers

one year ago