Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
I'M Trying To Use Store Some Data In Clearml-Data. Then I Want To Get It Back Elsewhere Using Dataset.Get_Local_Copy. It Returns A Directory But I Don'T Know How To Actually Access And Use That Data, And Remove It When It'S Done. Help Would Be Appreciated

I'm trying to use store some data in clearml-data. Then I want to get it back elsewhere using dataset.get_local_copy. It returns a directory but I don't know how to actually access and use that data, and remove it when it's done. Help would be appreciated.

  
  
Posted 2 years ago
Votes Newest

Answers 13


Here they are. I've created and published the dataset. Then when I try to get a local copy, the code works but i'm not sure how to proceed to be able to use that data.

  
  
Posted 2 years ago

VexedCat68 Hi 🙂

Can you please provide snippets of how you're saving and retrieving the files?

  
  
Posted 2 years ago

os.listdir(location) shows nothing

  
  
Posted 2 years ago

VexedCat68 , in the screenshot you provided it looks like the location is being printed. Did you check to see if something is there?

  
  
Posted 2 years ago

There's data when I manually went there. The directory was originally hidden my bad.

  
  
Posted 2 years ago

You can delete locally but it should not affect the remote data.

The data itself is stored in the fileserver. Whatever you do locally does not affect the remote storage, only when creating a new version the changes should be stored (Like when using 'clearml-data sync').

  
  
Posted 2 years ago

In case of non mutable copy you get a direct link to the cache folder, you should not change the content. In case of mutable it's a local copy for you to change or play around with, you can make any changes you want there 🙂

  
  
Posted 2 years ago

Anyway I could apparently delete things in the dataset from the local copy. Isn't it supposed to be immutable?

  
  
Posted 2 years ago

How would the two be different? Other than I can pass the directory to local mutable copy

  
  
Posted 2 years ago

That makes sense. But doesn't that also hold true for dataset.get_local_mutable_copy()?

  
  
Posted 2 years ago

To me it still looks like the only difference is that the non mutable copy is downloaded to the cache folder while mutable copy downloads to the directory I want. I could delete files from both sets so it seems like it's up to the user to make sure not to mutate the non mutable download in the cache folder.

  
  
Posted 2 years ago

I hope what I said was clear. Basically in reality they both seem mutable, with just the directory downloaded to being optional in one and in the other it's downloaded to the cache folder always.

  
  
Posted 2 years ago

I think that's about that 🙂

  
  
Posted 2 years ago
586 Views
13 Answers
2 years ago
one year ago
Tags