Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I Uploaded Direct Access File To Clearml Dataset System Like This One. How Can I Access The Link Of The Uploaded Item. Whenever I Try To Call

I uploaded direct access file to clearml dataset system like this one. How can I access the link of the uploaded item. Whenever I try to call get_local_file , the function copy the origional file to cached folder. How can I prevent copying and receive back the link of those file using Dataset function? Do you guys have any suggestion?

My purpose: my dataset is really large and I just want to use clearml to save the link of training data. I don't want to real uploading, downloading or caching of the training data 🙇
image

  
  
Posted one year ago
Votes Newest

Answers 7


Hi AdorableCrocodile14 ! get_local_copy will always copy/download external files to a folder. To get the external files, there is property on the dataset called link_entries which returns a list of LinkEntry objects, which contain a link attribute, and each such link should point to a extrenal file (in this case, your local paths prefixed with file:// )

  
  
Posted one year ago

SmugDolphin23 Great, I am able to do it now. Thank you so much 🙇
image

  
  
Posted one year ago

AdorableCrocodile14 any file starting with file:// is considered a direct access path by default

  
  
Posted one year ago

SillyLobster91 Here you go

  
  
Posted one year ago

Can you provide a code example of what you're doing?

  
  
Posted one year ago

I.e. how are you creating the dataset and how you are trying to retrieve it

  
  
Posted one year ago

SuccessfulKoala55 The following is how I create the dataset and how I am trying to retrieve it. Is there any other way to retrieve, without actually download dataset (copying) and use the direct link access

My cache folder is /mnt/ssd2t/clearml
image
image

  
  
Posted one year ago