hi, I created a dataset with 20K files, total of 20GB, with storage pointing to S3.
When I upload (or close) the dataset, during the compression phase, the clear-ml cli is killed due to OOM.
Compressing file1.mp4 Compressing file2.mp4 Compressing file3.mp4 . . . KilledThe machine has 16 GB memory, I could always define swap memory but crashing seems excessive