Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, I Have Quite A Generic Question. Basically, I Am Picking Your Brains For Any Solution. Our Current Pipeline Has (Clearml-Data, Clearml And Seldon). We Were Looking For Some Workflow Orchestrator To Stitch Them Up. One Scenario:

Hi, I have quite a generic question. Basically, I am picking your brains for any solution.
Our current pipeline has (clearml-data, clearml and seldon). We were looking for some workflow orchestrator to stitch them up.
One scenario:
Lets say after trying a few models. the data scientist has chosen one to publish. Using Clearml monitors, A notification is issued.. From here onwards, we want to have a workflow (like some customizable user workflow for approval). Maybe manager reviews the model, if it is Ok. He pushes it to the director. While director can choose to reject or push the approval process to someone else. This should be encapsulated in some workflow logic.
Our initial idea is to use airflow or prefect , however I was thinking whether if there is any other alternative business workflow engines which can do this better.

  
  
Posted 2 years ago
Votes Newest

Answers 12


we can add non-clearml code as a step in the pipeline controller.

Yes 🙂 , btw you can kind of already do that, with pre/post function callbacks (notice they are running from the same scope as the actual pipeline controller).
What exactly did you have in mind to put there ?

  
  
Posted 2 years ago

Hi AgitatedDove14 ,
At this point, Showing the url of the cleamltask might be sufficient. Unless in the future, someone wants it to be customised.
But the bigger question is if there is tool to aid with this workflow building ? We are currently experimenting with airflow/prefect.

  
  
Posted 2 years ago

Hi DeliciousBluewhale87
This sounds like a great workflow to implement.
I guess my first question is how do you imagine the manager/director interacting with the system? What will they be shown, to allow them to approve/decline the model promotion ?

  
  
Posted 2 years ago

Hi, we choose prefect ...

  
  
Posted 2 years ago

One use case now :
Load Data from Label Studio (Manager to manually approve) Push data to Clearml-data Run Training (Manager to manually Publish) Pushes model uri to next step Seldon deploy itLater, if seldon detects a data drift, it will automatically run (steps 2-5)..
At this point, we havent drilled all of it down yet

  
  
Posted 2 years ago

we also might have some other steps incorporated for other tools. We intend to have Label-Studio upstream.. So defintely needed some orchestrator tool

  
  
Posted 2 years ago

What do you already have working from the above steps ? and which parts are missing or we can think of automating ?

  
  
Posted 2 years ago

Ah, so in the future, we can add non-clearml code as a step in the pipeline controller.

  
  
Posted 2 years ago

DeliciousBluewhale87 what solution did you land on for this?

  
  
Posted 2 years ago

is that because you couldn’t find a good way to have a “manual approval/selection” step in http://clear.ml ?
Apart from that seems that pipeline task could have worked?

  
  
Posted 2 years ago

DeliciousBluewhale87 fyi, the new version of the pipeline (hopefully pushed towards the end of this week), will allow you to more easily write steps as functions (not only as Tasks, as the current implementation)
Also check the new Trigger and Scheduler both intended to trigger these pipelines:
https://github.com/allegroai/clearml/blob/fe3c481c37e70881c44d67c1cf9bbce00a84747e/clearml/automation/scheduler.py#L457
https://github.com/allegroai/clearml/blob/fe3c481c37e70881c44d67c1cf9bbce00a84747e/clearml/automation/trigger.py#L148

  
  
Posted 2 years ago

RoughTiger69
So prefect tasks :
Loads Data into clearml-data Runs trainining in clearml Publish model (manual trigger required, so user publishes model) and return model url Seldon deploys the model ( model url passed in)

  
  
Posted 2 years ago
205 Views
12 Answers
2 years ago
8 months ago
Tags