Sounds interesting. But my main concern with this kind of approach is if the surface of the (hparam1, hparam2, objective_fn_score) is non-convex, using your method you may not reach the best set of hyperparameters. Maybe try using smarter search algorithms, like BOHB or TPE if you have a large search space, otherwise, you can try to do a few rounds of manual random search, reducing the search space around the region of most-likely best hyperparameters after every round.

As for why structure your code using pipelines, I come from a somewhat heavy software engineering background, so for me a cleanly split codebase into components with clear responsibilities is the best thing, and caching is just a nice addition 🙂