Hi @<1720249421582569472:profile|NonchalantSeaanemone34>
Is it possible to read data directly from server w/o using get_local_copy()?
do you mean an artifact ? what is direct here?
Hi @<1523701205467926528:profile|AgitatedDove14>
What I mean is, dataset is loaded in server, I want to call that dataset in local PC without downloading its copy and do the analysis.
I want to call that dataset in local PC without downloading
when you say "call" what do you mean? the dataset itself is a set of files compressed and stored in the clearml file server (or on your S3 bucket etc.)
Please see this code snippet:
reviews_dataset_path = Dataset.get(
dataset_id=config["reviews_data_file_id"],
only_completed=True,
only_published=False,
). get_local_copy()
is there a way we can still read the data without using ". get_local_copy()"
No, it is zipped and stored, so in order to open the zipfile and read the files you have to download them.
That said everything is cached, so if the machine already downloaded the dataset there is zero download / unzipping,
make sese?
Thats nice to hear.
Thanks a lot @<1523701205467926528:profile|AgitatedDove14> :-)