Let'S Say I Have A Project Call Proj1 To Store Datasets With Type "Data Process".. What Is The Best Practice To Get The Latest Datasets ? Example, I Start The First Data (A). Then Using Clearml-Data, I Add Another Dataset (B) As Child To The Previous On

Unanswered

Hi DeliciousBluewhale87 ,

You can just get a local copy of the dataset with ds.get_local_copy() , this will download the dataset from the dataset task (using cache) and return a path to the downloaded files.

Now, in this path you’ll have all the files that you have in the dataset, you can go over the files in the dataset with ds.list_files() (or ds.list_files()[0] if you have only 1 file) and get the one you want

maybe something like:

ds_path = ds.get_local_copy() iris_pickle_file_name = ds.list_files()[0] iris_pickle_path = os.path.join(ds_path, iris_pickle_file_name)Can this do the trick?

  				
Posted 
	3 years ago

					More
				  		
  Report
		
					TimelyPenguin76
				
					0
					 Administrator

107 Views

0 Answers

3 years ago

one year ago