
Reputation
Badges 1
46 × Eureka!@<1523701070390366208:profile|CostlyOstrich36> Any idea please? We could use our 8xA100 as 8 workers, for 8 single-gpu jobs running faster than on a single 1xV100 each.
Oh? Worth trying!
Dang, so unlike screenshots, reports do not survive task deletion :/
(actually, that might even be feasible without touching the UI, depending how the plot is rendered, but I'll check)
Thanks @<1523703436166565888:profile|DeterminedCrab71> . Yes, I've seen the three options to plot different things. What I'm trying to do is for the "Iterations" plot to have the same plot but just change the X label, not the time series. In matplotlib that would be a call to xlabel
.
Can the “multiple agents on a single queue” scenario, combined with the autoscaler, spawn multiple agents on a single EC2 instance, by chance, please? (thinking e.g. 8 agents on a 8xGPU machine)
Tagging my colleague @<1529271085315395584:profile|AmusedCat74> who made that report.
Happy to jump on a call if easier to make sense of it :)
@<1523701087100473344:profile|SuccessfulKoala55> yes I am 🙂 And thanks, looking forward to it!
Hi 🙂 Anyone having any idea on that one please? Or could point me in the right place or the right person to find out? Thanks for any help!
The problem with logging as a 2D plot is we lose the streaming: if I understand correctly the documentation, Logger.current_logger().report_scatter2d
logs a single, frozen 2D plot when you know the full X and Y data. And you would do that at each evaluation step.
Logging scalars allows to log a growing time series, i.e. add to the existing series/plot at every "iteration", thus being able to monitor the progress over time in one single plot. It's a much more logical setting.
OK, so no way to have an automatic dispatch to different, correctly-sized instances, it’s only achievable by submitting to different queues?
Great, thanks both! I suspect this might need an extra option to be passed via the SDK, to save the iteration scaling at logging time, which the UI can then use at rendering time.
What is the best way to achieve that please?