PricklyRaven28 I think then you're looking for: ZipFile.ZIP_STORED
It's models not datasets in our case...
But we can also just tar the folder and return that... Was just hoping to avoid doing that
you can pass use the compression
parameter in dataset.upload
. The supported values are:ZipFile.ZIP_STORED (no compression) ZipFile.ZIP_DEFLATED (requires zlib) ZipFile.ZIP_BZIP2 (requires bz2) ZipFile.ZIP_LZMA (requires lzma)
Note that you need to import ZipFile
beforehand: from zipfile import ZipFile
You're probably looking for ZIP_BZIP2
, but I'm not sure about that.
Ok, tnx (:
We just see that taring and untaring is much faster than zip for big models
For pipelines there's currently no way to use different compressions. You can still use it when explicitly uploading https://clear.ml/docs/latest/docs/references/sdk/dataset/#upload
I know zip and tar.gz are supported for auto extraction. But you're looking for a setting to have artifacts compressed with tar instead of zip?