Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Unanswered
My Team Uses Metaflow By Outerbounds. Great Dag Tool. Super Robust. We Run Our Production Workloads On It And Use It For Experimentation, Too. I'M Considering Adding Clearml To Our Stack As An Exp Tracker / Model Registry Rather Than Going With The More


Like, if you google "dagster and clearml" or "prefect and clearml" or "airflow and clearml" -- I don't find any blogs written by people talking about how they use both of them together.

Oh yeah I see your point, I think the main reason is a lot of the dag capabilities and the orchestration is already folded into clearml's capabilities (i.e. pipelines + clearml-agent etc.)
That said I'm pretty sure I have seen just adding Task.init into each of a the framework above steps, in order to track the individual execution with higher degree of visibility (e.g. resource monitoring, artifacts, scalars etc).
One thing that you could do in order to also have some"dag" visibility inside clearml even when running dags with metaflow is using the "parent" property of the Task, point back to the parent Task in the dag, which means that you could trace back from each step the creating steps

So my point was: if ClearML can work well with Metaflow, it should be able to work well with any of the others, which I think would be great.

Correct, for example Sagemaker as a veri different example of dag/orchestration

And it also makes me wonder: why?? Why is it that seemingly nobody is using ClearML together with another DAG tool? Does it not make sense for some reason? Or is it that no one has explored it?

see my point above, pipelines/dags are already included, also supporting Logic not just dag, which allows for great flexibility

We've got some pressure internally to come up with something. The default is MLflow.

I think it's just missing some of the capabilities of ClearML, but diffidently a valid solution. If large scale is never a target, then for sure if it is easier and you do not mind too many solutions to manage.

  
  
Posted 2 months ago
39 Views
0 Answers
2 months ago
2 months ago