Thanks a lot AgitatedDove14 .
I'm thinking of a scenario that allows multiple users to quickly debug their GPU utilization, change relevant parameters, and re-run the training.
So your remarks about doing this in TB is correct since this is not relevant for all your experiments, but it could offer value as a quick solution.