Unanswered
Hi It Is Me Again, This Time Trying To Upload A Single File As Dataset But Met With The Following Error. The File Is 13.42Gb And Of Apache Arrow Format. Any Idea How To Solve This Error Please? Thank You.
total size 5.34 GB, 1 chunked stored (average size 5.34 GB)PanickyAnt52 The issue itself the Dataset will not break files (it will package into multiple zip files a large folder, but not break the file itself).
The upload itself is limited by the HTTP interface (i.e. 2GB file size limit)
I would just encode it into multiple Arrow files
does that make sense ?
101 Views
0
Answers
2 years ago
one year ago