Hello, I Look Up To The Clearml Data Documentation, The Dataset.Add_External_Files Method Seems Like To Perform A Download From The Source Url. What If The Case I Already Have My Dataset In The Output_Uri That I Specify In The Dataset.Create? Btw I Use A

Answered

Hello, I look up to the clearml data documentation, the Dataset.add_external_files method seems like to perform a download from the source url. What if the case I already have my dataset in the output_uri that I specify in the Dataset.create? Btw I use a network storage to store the dataset

  				
Posted 
	2 months ago

					More  		
  Report
		
					ConfusedSealion46
				
					0
					 × 1

Votes Newest

Answers 2

I think it is an issue of confusion here. I thought that using the add_external_files causes the clearml server to download the dataset to the server. After investigating through the file system, it turns out that the server save it as a link in state.json

  				
Posted 
	2 months ago

					More  		
  Report
		
					ConfusedSealion46
				
					0
					 × 1

Hi ConfusedSealion46 , in that case you can simply use add_external_files to the files that are already in your storage. Or am I missing something?

  				
Posted 
	2 months ago

					More  		
  Report
		
					CostlyOstrich36
				
					0

Write your answer

262 Views

2 Answers

2 months ago