Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi! I Have Question About Data Managment Part Of Clearml. Does Clearml Support Data Versioning Like In Lakefs ?) Is It Similar ? Maybe There Is Some Interesting Pros And Cons?

Hi!
I have question about data managment part of ClearML.

Does ClearML support data versioning like in LakeFS ?) Is it similar ? maybe there is some interesting pros and cons?

  
  
Posted 6 months ago
Votes Newest

Answers 4


yes) thanks)

  
  
Posted 6 months ago

Hi @<1523701205467926528:profile|AgitatedDove14>
Actually trying to understand) does clearml infrastructure better than common popular stack like dvc/lakefs + mlflow + cubeflow/airflow)

  
  
Posted 6 months ago

You mean does one solution is better than combining maintaining and automating 3+ solutions (dvc/lakefs + mlflow + cubeflow/airflow)
Yes I'd say it is. BTW if you have airflow running for other automations you can very easily combine the automation with clearml and have a single airflow automation for everything, but the main difference now airflow only launches logic, never actual compute/data (which are launched and scaled via clearml
Does that make sense?

  
  
Posted 6 months ago

Hi @<1697056701116583936:profile|JealousArcticwolf24>
You have clearml Datasets None
It will version catalog and store meta-data of your datasets.
Each version only stores the delta from the parent version, but delta is on a file granularity not a "block" granularity
Notice that under the hood of course it uses storage solutions to store and cache the underlying immutable copy of the data. What's your use case?

  
  
Posted 6 months ago
580 Views
4 Answers
6 months ago
6 months ago
Tags