Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I'M Looking For An Example Show How

Hi,
I'm looking for an example show how clearml dataset is consumed by sklearn pipeline or pytorch datapipe. e.g. since pytorch uses torch datasets, can we wrap clearml dataset to run as a torch dataset

Another interesting example would be how clearml dataset integrated with tfrecord

Any suggestions would be appricated

  
  
Posted 2 years ago
Votes Newest

Answers 4


Hi OutrageousSheep60 ,

I'm not sure I've seen such an example. But you can definately wrap torch dataset in clearml datasets. I am over simplifying but in the end think of the clearml dataset is like a large zip of all your files.

  
  
Posted 2 years ago

Hi OutrageousSheep60
Do you mean something like:
https://github.com/allegroai/clearml/tree/master/examples/datasets
?

  
  
Posted 2 years ago

OutrageousSheep60 , I think you can take the file that you get, pickle it or zip it and then simply save it as a clearml dataset. What do you think?

  
  
Posted 2 years ago

Possibly - thinking more of https://github.com/pytorch/data/blob/main/examples/vision/caltech256.py - using clearml dataset as root path.

  
  
Posted 2 years ago
1K Views
4 Answers
2 years ago
one year ago
Tags