Hi @<1661542579272945664:profile|SaltySpider22>
question 1: are parallel writes to a dataset with the same version possible?
When you are saying parallel what do you mean? from multiple machines ?
Whats the recommended way to append the dataset in a future version?
Once a dataset was finalized the only way to add files is to add another version that inherits from the previous one (i.e. the finalized version becomes the parent of the new version)
If you are worried about multiple versions, just like in git you have squeeze 🙂
passing Dataset artifacts between tasks seems to be not possible,
The correct way would be to pas the Dataset ID, then other task would simple get it with Dataset.get
No need to worry about re-download, everything is automatically cached.
Make sense ?