Hi @<1655744373268156416:profile|StickyShrimp60> , do you have any code that can reproduce this behavior?
So early/mid experiment run everything aligned fine?
Yes, early in the experiment everything seems fine, then updating begins lagging and coming in chucks before not updating at all anymore
Hi @<1523701070390366208:profile|CostlyOstrich36> - unfortunately not something that I can easily share I am afraid. I am using the hosted solution btw and I really believe I am just logging way too often, I think about once a second for some 12 hours..
How are the metrics being reported? directly via the Logger module or via automatical logging of some framework? Also how are iterations reported
I am logging directly via the Logger module using the report_scalar method, adding iterations parameter in every call. In my recent runs I am only logging every 30s and this does seem to remove the issue do not see the issue