Hmm, I don't know. I think, I saw this feature in 'manual' HPO code, but probably I'm wrong.

Hi @<1523701062857396224:profile|AttractiveShrimp45> , I think this is currently by design. How would you suggest doing multiple metric optimization - priority between metrics after certain threshold is met?

Could it be multiple metrics that were combined into a single metric later on? Before the optimizer?