kind of on the same topic, it would be very useful if some kind of verbosity will be enabled.. some kind of progress bar for get_top_experiments()

I mean to get top_tasks

AgitatedDove14 , I am referring to some generic HPO scenario where you define some HP space lets say:
param1 = np.linspace(lower_bound, upper_bound, n) param2 = np.linspace(lower_bound, upper_bound, n)then you run an optimization that samples this HP space,
For each trial a sample is pulled from the space, some experiment is performed and you get a score. Then to analyze the behavior of your objective you want to understand the relation between the params and objective score.
Then if you ...

AgitatedDove14 , for creating a dedicated function I would suggest also including the actual sampled point in the HP space. This would be the most common use case, and essentially the reason for running the HPO understanding the sensitivity of metrics with respect to hyper-parameters

I have a small question about the response structure, each of the metrics has this structure:
metric_id: { ... "value": 0.0006447011, "min_value": 8.6326945e-06, "max_value": 0.001049518, ... } what does value refer to? the last reported?

it seem to be orders of magnitude faster!

that was never a problem.. before the new copy wasn't uploaded.. I'm still not sure what you referring to

or actually the local html, I believe it should work for a mounted s3

in order to have an automatically updating preview

thanks, I'll try this. Is there an efficient way to get the IDs first?

the report media workaround I suggested was referring to having the preview directly in the debug_samples of the given task

Thanks AgitatedDove14 , we have no issue with "html stuck in cache" I'm not sure what are you referring to..

so if I wait a few minutes I will be able to get the latest version?

CostlyOstrich36 , any thoughts about this one by the way?

after poking the setup in multiple ways we came to a conclusion that the API server is being clogged by calls from multiple HPOptimziners, and it utilizes a single core so it seems like we are not able to scale it up properly... any ideas?

How can we increase the number of API server processes?

optimizer.get_top_experiments(n)

FrothyDog40 , thoughts?

AgitatedDove14 it ended up not solving our problem.. it was a false observation.. there is some bottleneck that kills the UI responsiveness that we can't identify

AgitatedDove14 thanks, at peak usage we have 6-8 gb of free RAM

that would be great, thanks a lot!

just making sure, the services queue is available from the clearml-server UI

AgitatedDove14 after a small test run we done, we get some issues with the proposed approach
I think it works as far as what you mentioned is concerned, there is a dynamic link in the debug samples that downloads the html file when we try to view it, and it is updated according to the notebook
However we access our server from a limited networking machine that can only access AWS ips, and the HTML file is not a standalone and has some dependencies that require networking.. so can only be vi...

for me at the moment it means "manually" filtering the keys I've put in for the HP space. I find it a bit strange that they are not saved as part of the optimizer object..
the optimizer_task seem to have an attribute called hyper_parameters but its empty in my case..

AgitatedDove14 seem to work in terms of updating the file which is great! the notebook HTML preview seem not to work though.. I guess you are aware of it because the displayed text is saying something like click the link to view the file

https://github.com/allegroai/clearml/issues/488

so actually in our case we have an S3 link, but when downloading it, it seems correct.. there is probably some way to make an S3 path open up in the browser by default

FrothyDog40 , is submitting an issue still relevant?

AnxiousSeal95 , the roaming people, how can I find them actually?

Bonus 2: having a way to push experiments up in the queue