
Reputation
Badges 1
46 × Eureka!It was a debugging session. We haven’t yet tried a “Standard” non-debugging clearml session.
Great, thanks both! I suspect this might need an extra option to be passed via the SDK, to save the iteration scaling at logging time, which the UI can then use at rendering time.
(apologies for delay @<1523701087100473344:profile|SuccessfulKoala55> , we got called into meetings. Really appreciate your reactivity!)
(do you welcome PRs?)
Thanks @<1523703436166565888:profile|DeterminedCrab71> . Yes, I've seen the three options to plot different things. What I'm trying to do is for the "Iterations" plot to have the same plot but just change the X label, not the time series. In matplotlib that would be a call to xlabel
.
The problem with logging as a 2D plot is we lose the streaming: if I understand correctly the documentation, Logger.current_logger().report_scatter2d
logs a single, frozen 2D plot when you know the full X and Y data. And you would do that at each evaluation step.
Logging scalars allows to log a growing time series, i.e. add to the existing series/plot at every "iteration", thus being able to monitor the progress over time in one single plot. It's a much more logical setting.
@<1523701205467926528:profile|AgitatedDove14> great! (I'm on the Pro version :) ).
Thanks. That would be very helpful. Some of our graphs are logged by optimization steps, whereas some by epochs, so having all called "Iterations" is not ideal.
(actually, that might even be feasible without touching the UI, depending how the plot is rendered, but I'll check)
Brilliant, thanks a lot for the answer Jake, much appreciated and clearer!
@<1529271085315395584:profile|AmusedCat74> @<1548115177340145664:profile|HungryHorse70> here we have the answer :)
Oh? Worth trying!
Tagging my colleague @<1529271085315395584:profile|AmusedCat74> who made that report.
Is the doc on GitHub so we can copy that into a PR?
What is the best way to achieve that please?
No problem 🙂 Once you’ve merged it, what do we need to do to get the updated version please?
@<1523701087100473344:profile|SuccessfulKoala55> I think you’ve been tagged in the PR 🙂
Thanks @<1523701087100473344:profile|SuccessfulKoala55> ! Any inkling on how soon? Is it days, weeks, or months please? 🙂
cc my colleagues @<1529271085315395584:profile|AmusedCat74> and @<1548115177340145664:profile|HungryHorse70>
This great tool is worth paying for!
OK, so no way to have an automatic dispatch to different, correctly-sized instances, it’s only achievable by submitting to different queues?
Happy to jump on a call if easier to make sense of it :)
Yes, exactly. Here is the logical sense it makes: I have plots where iterations represent different units: for some these plots iterations (call them A) are optimization steps, while for others (call them B) they are evaluation iterations, occuring every N optimization steps. I would like to either:
- Change the X label so these different plots do not have the same label when they represent different things.
- Or, even better, keep the unique "iterations" label but be able to change how I lo...
@<1523701087100473344:profile|SuccessfulKoala55> yes I am 🙂 And thanks, looking forward to it!
@<1523701070390366208:profile|CostlyOstrich36> Any idea please? We could use our 8xA100 as 8 workers, for 8 single-gpu jobs running faster than on a single 1xV100 each.
Tagging @<1529271085315395584:profile|AmusedCat74> my colleague with whom we ran into this issue.