Hi All, Quick Question About Clearml Datasets. Does Anyone Know If It Is Possible To Access (Could Just Be Paths To The Data In A Bucket) A Dataset Directly From S3, Instead Of Downloading A Local Copy? We Typically Store And Access Large Quantities Of

Unanswered

Hi @<1686909730389233664:profile|AmiableSheep6> , I could suggest using the StorageManager module to pull specific files from S3.

There is no option to download specific files from a dataset. I would suggest breaking it into maybe smaller versions.

You would however need to pull the data locally for training anyways, wouldn't breaking it into smaller versions help this issue?

  				
Posted 
	one month ago

					More
				  		
  Report
		
					CostlyOstrich36
				
					0

18 Views

0 Answers

one month ago