Unanswered
			
			
 
			
	
		
			
		
		
		
		
	
			
		
		Two Questions About Datasets:
Question 1: Are Parallel Writes To A Dataset With The Same Version Possible? Is The Way To Go, To Have A Task, Which Creates A Dataset Object, Which In Turn Is Passed As Artifact To The Subsequent Ingestion Tasks? After The P
yes, or (because I deployed clearml using helm in kubernetes) from the same machine, but multiple pods (tasks).
Oh now I see, long story short, no  😞  the correct way of doing that is every node/pod creates it's own dataset,
then when you are done, you create a new version with the X datasets that you created as parents, the newly created version is just "meta" it basically tells the system how to combine the previously generated datasets (i.e. no data is actually re-uploaded).
Version tree should look something like
 [x]
  |
+-+--+---+
|    |   |
[a] [b] [c]
		284 Views
				0
Answers
				
					 
	one year ago
				
					
						 
	one year ago