@<1523701070390366208:profile|CostlyOstrich36>
Sorry for delay, please see below:
In the meantime, it might help to limit the amount of jobs using the advanced settings. If you know the exact amount and want to do every one for sure, just set it that way 🙂
Hi @<1523701062857396224:profile|AttractiveShrimp45> , can you please share some screenshots of what you see and also share a code snippet of what reproduces this behavior?
Hi @<1523701062857396224:profile|AttractiveShrimp45> , I'm checking your issue myself. Do you see any duplicate experiments in the summary table?
Ok, so I recreated your issue I think. Problem is, HPO was designed to handle more possible combinations of items than is reasonable to test. In this case though, there are only 11 possible parameter "combinations". But by default, ClearML sets the maximum amount of jobs much higher than that (check advanced settings in the wizard).
It seems like HPO doesn't check for duplicate experiments though, so that means it will keep spawning experiments (even though it might have executed the exact same one before) until either its job budget, time budget or iterations budget is reached.
I think this is a bug, or should at least be looked at, @<1523701062857396224:profile|AttractiveShrimp45> Do you mind opening a Github issue for this, so we can track it? 🙂
Below is a screenshot where indeed 2 of the same experiments were spawned
In this case though, there are only 11 possible parameter "combinations". But by default, ClearML sets the maximum amount of jobs much higher than that (check advanced settings in the wizard).
I could understand if it will use the minimum amount of jobs, not maximum 🙂
BTW, I created another HPO app with two parameters. And instead of 11*6=66 jobs I saw 92.
I'll open a bug.
@<1523701118159294464:profile|ExasperatedCrab78> - many thanks!
There is no code snippet, I created HPO app via UI.