
Reputation
Badges 1
46 × Eureka!From the doc I seemed to find ways to log 2D scatter plots, but not line plots :/ (found)
It also seems simpler to keep the scalar logging structure, but be able to pass a multiplier (reflecting the eval_n_steps
in for example Torch Lightning)
Does that make sense?
Logging scalars also leverages ClearML automatic logging. One problem is that this automatic logging seems to keep its own internal "iteration" counter for each scalar, as opposed to keeping track of, say, the optimizer's number of steps.
That can be simply fixed on clearML python lib by allowing to set a per-scalar iteration-multiplier.
Dang, so unlike screenshots, reports do not survive task deletion :/
Can the “multiple agents on a single queue” scenario, combined with the autoscaler, spawn multiple agents on a single EC2 instance, by chance, please? (thinking e.g. 8 agents on a 8xGPU machine)
Thanks @<1523701070390366208:profile|CostlyOstrich36> ! I'll do - and might even peek under the hood see if I can make a PR. What's the best repo for that? Is it that of the ClearML python package?
Brilliant, thanks a lot for the answer Jake, much appreciated and clearer!
@<1529271085315395584:profile|AmusedCat74> @<1548115177340145664:profile|HungryHorse70> here we have the answer :)
Is the doc on GitHub so we can copy that into a PR?
Hi 🙂 Anyone having any idea on that one please? Or could point me in the right place or the right person to find out? Thanks for any help!
And yes, I was also referring to tasks ran by the Autoscaler (potentially via the HPO) app, too.
Do Pipelines work with Hyperparameter search, and with single training jobs?
Thanks @<1523701070390366208:profile|CostlyOstrich36> !
- I hadn’t found the multiple-resources within the same autoscaler. Could you point me to the right place please? Are they all used interexchangeably based upon availability, rather than based on job needs?
- We thought of using separate queues (we do that for CPU vs GPU queues), but having ClearML automatically dispatch to the right based on a job specification would be more flexible. (for example, we could then think to dispath dynami...
Yes, we love the HPO app, and are using it :)