Can the user be overwritten during task configuration (I don't see such an option in the documentation)?

Hm, not really 😞 this is tied with security feature on top.
That said,

stored them as a k8s secret and they are reused whenever anyone from our ML team starts a new ML model training

Does that mean you are running an Agent on the k8s cluster? what's exactly the flow that causes your k8s credentials to be used