Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Hi, I'M Trying To Understand If Clearml Supports My Usecase: I Generate My Own Data (Problem-Specific) And Generation Process Is Uploading A Parquet Dataset (In Fact, Parquet Contains Multiple Files Obv.) To S3. Is There A Way To "Register" The Dataset In

Hi, I'm trying to understand if clearml supports my usecase: i generate my own data (problem-specific) and generation process is uploading a Parquet dataset (in fact, parquet contains multiple files obv.) to S3. Is there a way to "register" the dataset in Clearml without doing local copy (dataset is > 300GB)?

Posted 2 years ago
Votes Newest

Answers 3

+1 to this question

Posted 2 years ago

Thanks, didnt check here for a while, i have managed to find out about this myself but Thx anyway:)

Posted 2 years ago

Hi GreasyWalrus57 , sorry but didn’t get that.

You want to register the data? you can do it with clearml-data and then use this task to connect between tasks and data

Posted 2 years ago
3 Answers
2 years ago
8 months ago