(do you welcome PRs?)

Hi 🙂 Anyone having any idea on that one please? Or could point me in the right place or the right person to find out? Thanks for any help!

Dang, so unlike screenshots, reports do not survive task deletion :/

OK, so no way to have an automatic dispatch to different, correctly-sized instances, it’s only achievable by submitting to different queues?

Can the “multiple agents on a single queue” scenario, combined with the autoscaler, spawn multiple agents on a single EC2 instance, by chance, please? (thinking e.g. 8 agents on a 8xGPU machine)

Thanks @<1523701070390366208:profile|CostlyOstrich36> ! I'll do - and might even peek under the hood see if I can make a PR. What's the best repo for that? Is it that of the ClearML python package?

@<1523701205467926528:profile|AgitatedDove14> great! (I'm on the Pro version :) ).

Done: None

We’re on the PRO 🙂

@<1523701087100473344:profile|SuccessfulKoala55> I think you’ve been tagged in the PR 🙂

Is the doc on GitHub so we can copy that into a PR?

Yes, exactly. Here is the logical sense it makes: I have plots where iterations represent different units: for some these plots iterations (call them A) are optimization steps, while for others (call them B) they are evaluation iterations, occuring every N optimization steps. I would like to either:

• Change the X label so these different plots do not have the same label when they represent different things.
• Or, even better, keep the unique "iterations" label but be able to change how I lo...

What is the best way to achieve that please?

(actually, that might even be feasible without touching the UI, depending how the plot is rendered, but I'll check)

Thanks @<1523701087100473344:profile|SuccessfulKoala55> ! Any inkling on how soon? Is it days, weeks, or months please? 🙂

@<1523701070390366208:profile|CostlyOstrich36> Any idea please? We could use our 8xA100 as 8 workers, for 8 single-gpu jobs running faster than on a single 1xV100 each.

Yes

Happy to jump on a call if easier to make sense of it :)

From the doc I seemed to find ways to log 2D scatter plots, but not line plots :/ (found)
It also seems simpler to keep the scalar logging structure, but be able to pass a multiplier (reflecting the eval_n_steps in for example Torch Lightning)

Brilliant, thanks a lot for the answer Jake, much appreciated and clearer!
@<1529271085315395584:profile|AmusedCat74> @<1548115177340145664:profile|HungryHorse70> here we have the answer :)

Amazing, thanks a lot @<1523701087100473344:profile|SuccessfulKoala55> !

The problem with logging as a 2D plot is we lose the streaming: if I understand correctly the documentation, Logger.current_logger().report_scatter2d logs a single, frozen 2D plot when you know the full X and Y data. And you would do that at each evaluation step.
Logging scalars allows to log a growing time series, i.e. add to the existing series/plot at every "iteration", thus being able to monitor the progress over time in one single plot. It's a much more logical setting.

And yes, I was also referring to tasks ran by the Autoscaler (potentially via the HPO) app, too.

(apologies for delay @<1523701087100473344:profile|SuccessfulKoala55> , we got called into meetings. Really appreciate your reactivity!)

No problem 🙂 Once you’ve merged it, what do we need to do to get the updated version please?

We were just trying to run a plain clearml-session on the AWS autoscaler.

Do Pipelines work with Hyperparameter search, and with single training jobs?

Oh? Worth trying!

Thanks. That would be very helpful. Some of our graphs are logged by optimization steps, whereas some by epochs, so having all called "Iterations" is not ideal.

Tagging @<1529271085315395584:profile|AmusedCat74> my colleague with whom we ran into this issue.