Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello! When I Squash Multiple Datasets (E.G.

Hello! When I squash multiple datasets (e.g. Dataset.squash(dataset_name="new_ds", dataset_ids=[id1, id2, id3]) , as far as I can see the newly created dataset does not track which datasets where squashed. Do I have to add that information manually via e.g. Dataset.set_description, or is there another way to track that info? This would be important for data lineage reasons I think. Thanks!

  
  
Posted one year ago
Votes Newest

Answers 3


Ah, I wasn’t aware this is possible! Yes, perfect, thanks a lot!

  
  
Posted one year ago

And another question regarding squashing: sometimes I get the following error: FileNotFoundError: [Errno 2] No such file or directory: '/home/vscode/.clearml/cache/storage_manager/datasets/ds_4f3436f7b3ef484f8148a9c25a444ee5/file.ann — why is there an attempt to access the file locally?

  
  
Posted one year ago

Hi SmallGiraffe94 ! Dataset.squash doesn't set as parents the ids you specify in dataset_ids . Also, notice that the current behaviour of squash is pulling the files from all the datasetes from a temp folder and re-uploading them. How about creating a new dataset with id1, id2, id3 as parents Dataset.create(..., parent_datasets=[id1, id2, id3]) instead? Would this fit your usecase?

  
  
Posted one year ago
360 Views
3 Answers
one year ago
10 months ago
Tags