. I wonder if I can extend this to reporting grad_norm per layer.

oh that makes sense, technically I assume so, is this a HF logger option? notice ClearML is already integrated with HF on the HF side, do they report that when TB logger is used?