Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Community :) I'M New To Clearml And Seeking Advice On Best Practices For Managing Datasets. I Have Two Types Of Datasets: (1) Pdfs (2) Tabular Data Stored In Excel.

Hi community :)
I'm new to ClearML and seeking advice on best practices for managing datasets. I have two types of datasets:
(1) PDFs
(2) Tabular data stored in Excel.

Question 1: Tracking changes in different versions of Excel files
I frequently update my Excel datasets by adding new data and deleting old entries. Can I track these changes across different versions in ClearML?

For instance, if I upload an initial Excel file and later make modifications, is there a way to compare the versions to see what data was added or removed?

Question 2: Handling data stored in S3 without storing it in ClearML
I have a bucket in S3 that stores PDFs, and I prefer not to store these files directly in ClearML. Is there a way to track changes to the files in this S3 bucket, such as monitoring which files have been added or removed?

Thanks!

  
  
Posted 3 months ago
Votes Newest

Answers 2


I see, thanks for that!

  
  
Posted 3 months ago

Hi @<1724960475575226368:profile|GloriousKoala29> , to address your questions:

  • No, that is not possible currently. Think of the Datasets feature as a catalogue of data, meaning you can see what data is saved but you can only see what's inside when you pull it locally.
  • I'm afraid not, ClearML basically saves links to the data but doesn't directly "look" at the data
  
  
Posted 3 months ago
317 Views
2 Answers
3 months ago
3 months ago
Tags
Similar posts