Hi I am creating a custom image dataset and does anyone know how to upload files into remote uri (S3) without compressing the data? from clearml import Datas...
4 months ago
Which is not an ideal solution but for now it's working.
@<1706116294329241600:profile|MinuteMouse44> unfortunately no, I created my own upload method with python Django and passed the S3 directories to the clearML dataset to track of datasets.
I think based on the src file dataset.py
the compression False is not possible
def upload(
self,
show_progress=True,
verbose=False,
output_url=None,
compression=None,
chunk_size=None,
max_workers=None,
retries=3,
):
# type: (bool, bool, Optional[str], Optional[str], int, Optional[int], int) -> ()
"""
Start file uploading, the function returns when all files are uploaded.
:param sho...