Can Anyone Help Me To Explain The Differences Between

Answered

Can anyone help me to explain the differences between path_substitution and direct_access . As I understand:

direct_access : register data will not be pushed to ClearML server.Dataset.create then upload => Server only save the local path of the file.
Dataset.get => server will return the local path url
path_substitution : is it like symlink from local link to fake clearml_server link. I still don't get the idea how the server and local user interact hereIs my understanding correct? Can anyone help me to clarify this?

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AdorableCrocodile14
				
					0
					 × 1

Votes Newest

Answers 3

@<1523701087100473344:profile|SuccessfulKoala55> Is there any way .get_local_copy return direct link of those local files items. Data is duplicated. Those data existed in local storage, but they are still needed to save in cached folder

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AdorableCrocodile14
				
					0
					 × 1

Hi @<1570583237065969664:profile|AdorableCrocodile14> , I think this is a good answer: None

  				
Posted 
	one year ago

					More
				  		
  Report
		
					SuccessfulKoala55
				
					0
					 × 1

can you give me an example of use of direct_access setup 🙇

I have large dataset, would like to register the dataset in ClearML without uploading real dataset. When running a task, I can get the dataset without creating a cache folder

  				
Posted 
	one year ago

					More
				  		
  Report
		
					AdorableCrocodile14
				
					0
					 × 1

Write your answer

887 Views

3 Answers

one year ago