I also get this from the logging
TRAINS Monitor: GPU monitoring failed getting GPU reading, switching off GPU monitoring
it's ok, I don''t think this is very important. thx
Hi @<1523713917677735936:profile|EnviousStarfish54> ,
Trains samples the usage every 30 seconds, and reports on every iteration (if no iterations are reported, it should be reported every 3min approximately). The error you see indicates that Trains failed to detect stats for a GPU on your machine, in which case it should still report all the other stats (CPU etc.)
Currently, no - is that a real use-case in your opinion?
Hi @<1523713917677735936:profile|EnviousStarfish54> ,
In your experiment's Results/Scalars
sub-panel you should see a graph titled :monitor:machine
which contains a series per monitored value (e.g. CPU usage etc.)
How often does Trains record the usage? I tried running a dummy programming which sleeps for 20 seconds and I don't see any plot.
Is this configurable?
@<1523701087100473344:profile|SuccessfulKoala55> Thx! I swear I saw it before, but somehow I just overlooked it.
Is it possible to set the frequency of sampling?