Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Unanswered
I Saw Some Talk Of Clearml + Kedro On Reddit. Is That A Good Approach?


one can containerise the whole pipeline and run it pretty much anywhere.

Does that mean the entire pipeline will be running on the instance spinning the container ?
From here: this is what I understand:
https://kedro.readthedocs.io/en/stable/10_deployment/06_kubeflow.html

My thinking was I can use one command and run all steps locally while still registering all "nodes/functions/inputs/outputs etc" with clearml such that I could also then later go into the interface and clone any of the individual steps and run them again.

That is absolutely correct 🙂

With another command I could also just pseudo run the pipeline with kedro locally to register everything in clearml and then run it on a clearml agent.

Sure this will work

I thought that in both cases I would need to create a PipelineController Task at the end with the full pipeline included I could even just clone that one.

This is exactly how the pipeline is designed, and cloning and running the pipeline controller should work and launch the entire pipeline (usually the controller is executed on the services queue, and the pipeline Tasks are launched on a GPU or a CPU queue)

If I want to use a hook system (e.g. kedro provides hooks for running callbacks before and after nodes/tasks)

Yes I'm with you I think this is the main challenge here.

Is there a way to use Task as a decorator on a function level?

Yep, this is exactly what's coming in the next release of Pipelines (RC should be out in a week or so)

  
  
Posted 3 years ago
147 Views
0 Answers
3 years ago
one year ago