Reputation
Badges 1
6 × Eureka!Hi Eugen
Thanks for the response!
let me know if you need a running example, that will require some effort to produce.
It just puzzles me that if a single subprocess spawns a task, meaning that only one task is active, and that task is only accessed by the worker creating it, the massive slowdown happens
It seems that running Task.init() outside in the main process before spawning multiple processes, and calling Task.current_task() inside the rank = 0 process works with no slowdown
With the logger.report_scalar
Hi @<1523701087100473344:profile|SuccessfulKoala55> We wonder if this is intended behavior from clearml? That the longer the experiment runs, it start averaging over datapoints. We have had experiments where the variance in the datapoints were significant, and this "feature" meant that our variance would become smaller and smaller?
It seems that the longer it runs, and the more datapoints it accumulates, it starts averaging over iterations