I want to call that dataset in local PC without downloading
when you say "call" what do you mean? the dataset itself is a set of files compressed and stored in the clearml file server (or on your S3 bucket etc.)
Hi @<1523701205467926528:profile|AgitatedDove14>
What I mean is, dataset is loaded in server, I want to call that dataset in local PC without downloading its copy and do the analysis.
Please see this code snippet:
reviews_dataset_path = Dataset.get(
dataset_id=config["reviews_data_file_id"],
only_completed=True,
only_published=False,
). get_local_copy()
is there a way we can still read the data without using ". get_local_copy()"
Thats nice to hear.
Thanks a lot @<1523701205467926528:profile|AgitatedDove14> :-)
No, it is zipped and stored, so in order to open the zipfile and read the files you have to download them.
That said everything is cached, so if the machine already downloaded the dataset there is zero download / unzipping,
make sese?
Hi @<1720249421582569472:profile|NonchalantSeaanemone34>
Is it possible to read data directly from server w/o using get_local_copy()?
do you mean an artifact ? what is direct here?