I know I drop a very general question, so I am sorry for the confusion.

However, I was reading online how other companies were dealing with data pipeline for handling data both in training & inference in a very standard, and monitorable way.

I saw people refer to something called feature store , the first was probably uber ( source https://eng.uber.com/michelangelo-machine-learning-platform/ ) ... I know tecton is one provider of it (source https://www.tecton.ai/ ), but there might be others.

Does this clarify a bit more?