Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Folks, We Are Trying To Find A Tool To Help With Workflow Orchestration. This Is Our Stack So Far (Label Studio/Clearml/Seldon). Does Anyone Have Any Experience With Using Any Workflow Which Is Most Compatible Esp Wrt To Clearml.

Hi folks, we are trying to find a tool to help with workflow orchestration. This is our stack so far (Label Studio/Clearml/Seldon). Does anyone have any experience with using any workflow which is most compatible esp wrt to Clearml.
https://neptune.ai/blog/best-workflow-and-pipeline-orchestration-tools

  
  
Posted 2 years ago
Votes Newest

Answers 12


Hi DeliciousBluewhale87
When you say "workflow orchestration", do you mean like a pipeline automation ?

  
  
Posted 2 years ago

Yeah within clearml , we use the PipelineController. We are now mainly looking for a single tool to stitch together other products.
But of course, will give first precedence to tools which will work best with clearml. Thus asking, if anyone has had similar experience on setting up such systems.

  
  
Posted 2 years ago

Still figuring out, what is the best orchestration tool,which can run this end-2-end.

DeliciousBluewhale87 / PleasantGiraffe85 based on the scenario above what is the missing step that you need to cover? Is it the UI presenting the entire workflow? Or maybe the a start trigger that can be configured ?

  
  
Posted 2 years ago

TenseOstrich47 / PleasantGiraffe85
The next version (I think releasing today) will already contain scheduling, and the next one (probably RC right after) will include triggering. That said currently the UI wizard for both (i.e. creating the triggers), is only available in the community hosted service. That said I think that creating it from code (triggers/schedule) actually makes a lot of sense,

pipeline presented in a clear UI,

This is actually actively worked on, I think AnxiousSeal95 would love to get your feedback on such things, your opnion actual matters a lot!

  
  
Posted 2 years ago

We are planning on using airflow as the orchestration, but that may not fit your needs. I would say that the tool of choice is highly context specific.

We will be using airflow to trigger clearml-defined pipelines based on 'trigger' events, such as degradation in model performance, error alerts (e.g. at the data transformation task) etc.

  
  
Posted 2 years ago

Sure, with pleasure. However, we're using a self-hosted (on premise) version of ClearML...

  
  
Posted 2 years ago

AgitatedDove14 We too self host (on prem) the helm charts in our local k8s ecosystem.
Triggering - Will be nice feature indeed, currently we are using clearml.monitors to address these now
Is it the UI presenting the entire workflow? - This portion will also be nice. (Let's say someone uses a 1) clearmldataset -> 2) Pipeline Controller (Contains preprocessing, training, hyperparamter tuning) -> 3) clearml-serving ).. If they can see the entire thing, in one flow
We are using seldon for other reasons, thus cant use clearml-serving but at least the first 2 components should be common for most folks.

  
  
Posted 2 years ago

image

  
  
Posted 2 years ago

AgitatedDove14 Not creating but more for orchestrating...

Currently, we manually push a dataset to cleaml-dataset .
Have a pipeline controller Task which (takes in data from clearml-dataset, runs preprocessing, runs training) and Publishes a model (if certain threshold is met).
We have clearml monitor which will monitor all Published models .It will push the uri of the published model to a rabbitmq.

We have a subscriber (python code) listening to the rabbitmq. This takes in the uri from the queue and creates a seldon deployment.

Here the vast majority of the stuff gets done in clearml. Still figuring out, what is the best orchestration tool,which can run this end-2-end.

  
  
Posted 2 years ago

For us it is both - having the process/pipeline presented in a clear UI, and the ability to trigger it, e.g. every evening.
In addition, tools like Dagster offer code-organization, and a separation of the code itself from the data and the configuration. So that we can use the same data/ml pipeline for different use-cases.

  
  
Posted 2 years ago

I'm also currently in a similar process, and giving a shot to http://DAGster.io

  
  
Posted 2 years ago

So like a UI for creating pipelines doing different things on the different solutions ?

  
  
Posted 2 years ago
678 Views
12 Answers
2 years ago
one month ago
Tags