Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Guys! Broad Question: Do You Work With A Feature Store Along With Clearml? Right Now My Workflow Is: 1) Clearml Task Fetches Data From Database/Api, Does A Rough Cleaning And Saves A "Raw" Versioned Dataset On Clearml 2) Another Clearml Task Trains A

Hi guys! Broad question: do you work with a feature store along with ClearML?

Right now my workflow is:

  1. clearml task fetches data from database/api, does a rough cleaning and saves a "raw" versioned dataset on clearml
  2. another clearml task trains a model on the latest version of the dataset and stores an an artifact. The model artifact takes as input raw data, and has methods for doing both preprocessing / feature creation and inference.
  3. external services (atm just a couple fastapi apps) load the artifact, take in raw data, do inference and all that stuff.

Do you have a better way of handling feature transformation?

  
  
Posted 6 months ago
pic
More
FlutteringMouse14
0 × 1
Report
Votes Newest

Answers 4

I can imagine :D I'll look into it, thanks!

  
  
Posted 6 months ago
pic
More
FlutteringMouse14
0 × 1
Report

Exactly!
Regarding adding feature store, probably not in the near future, a scalable feature store is quite the project, probably more realistic to somehow have a recipe to deploy with Feast

  
  
Posted 6 months ago
pic
More
AgitatedDove14
0 × 1
Report

Cool thanks! So for online Feature transformation I would set up a feast server, and the tasks would use feast to create and retrieve features, right?

I know it's a huge project, but it would be great to have a feature store as part of ClearML, do you have any plans to do that?

  
  
Posted 6 months ago
pic
More
FlutteringMouse14
0 × 1
Report

Hi @<1691620877822595072:profile|FlutteringMouse14>
Yes, feast has been integrated by at least a couple if I remember correctly.
Basically there are two ways offline and online feature transformation. For offline your pipeline is exactly what would be recommended. The main difference is online transformation where I think feast is a great start

  
  
Posted 6 months ago
pic
More
AgitatedDove14
0 × 1
Report
Write your answer
709 Views
4 Answers
6 months ago
6 months ago
Tags
clearml