AgitatedDove14 thanks, I actually experimented with similar parallel pool approach but the overhead seem to even out the benefit..
is there something you can think of for the first part though? pulling all the experiments get_top_experiments()

AgitatedDove14 , what I meant by manually filtering, at the moment, to combine the information of metric values + HP point, I pull all the parameters, and then manually filter on the HP keys (manually=I have to plug them in, they are not part of optimizer object)

that is the heaviest part for me

for me at the moment it means "manually" filtering the keys I've put in for the HP space. I find it a bit strange that they are not saved as part of the optimizer object..
the optimizer_task seem to have an attribute called hyper_parameters but its empty in my case..

I have a small question about the response structure, each of the metrics has this structure:
metric_id: { ... "value": 0.0006447011, "min_value": 8.6326945e-06, "max_value": 0.001049518, ... } what does value refer to? the last reported?

SuccessfulKoala55 , yes, that part is 100% correct

but I don't get the same when I try to reproduce it for debug as described above (with task.connect)

this is what I get with curl

sounds great, is it part of the release? or latest repo?

one more question, is there a way to assign a job to a specific worker? or is it only working on queue level

I have the following flow:
create a task draft on clearml UI Enqueue it agent picks up the task Agent synchronizes (some custom mechanism I have) the hyper parameters of the experiment with some local config object Agent runs task

I was hoping for something that I can scale

it doesn't even need to be a sub process at this point.. it can be serial execution

FrothyDog40 , done 🙂
https://github.com/allegroai/clearml/issues/474

client has the following attributes:
['auth', 'events', 'models', 'projects', 'queues', 'session', 'tasks', 'workers']

kind of on the same topic, it would be very useful if some kind of verbosity will be enabled.. some kind of progress bar for get_top_experiments()

AgitatedDove14 no it has an offset of the value that it started with, so for example you stopped at n, then when you are running the n+1 epoch you get the 2*n+1 reported

the report media workaround I suggested was referring to having the preview directly in the debug_samples of the given task

AgitatedDove14 after a small test run we done, we get some issues with the proposed approach
I think it works as far as what you mentioned is concerned, there is a dynamic link in the debug samples that downloads the html file when we try to view it, and it is updated according to the notebook
However we access our server from a limited networking machine that can only access AWS ips, and the HTML file is not a standalone and has some dependencies that require networking.. so can only be vi...

AgitatedDove14 seem to work in terms of updating the file which is great! the notebook HTML preview seem not to work though.. I guess you are aware of it because the displayed text is saying something like click the link to view the file

Thanks AgitatedDove14 , we have no issue with "html stuck in cache" I'm not sure what are you referring to..

Hi AgitatedDove14 , so it looks something like this:
` Task.init
trainer.fit(model) # clearml logging starts from 0 and logs all summaries correctly according to real count

triggered fit stopping at epoch=n

something

trainer.fit(model) # clearml logging starts from n+n (thats how it seems) for non explicit scalar summaries (debug samples, scalar resources monitoring, and also global iteration count)

triggered fit stopping

... `I am at the moment diverging from this implementation to s...

AgitatedDove14 , definitely so, this is very generic and very useful
In many cases the objective is just one of multiple metrics of interest, so for me almost always I would want to combine it with the rest of the scalar metrics

The "notebook preview" link doesn't pop-up the s3 credentials, however the "notebook" artifact when clicking on the download button does show this pop up
they actually seem to be different in the way that the UI server is showing them.. one is a link (the html) and has no pop up, the other one (.ipynb) is a text with a download button in the end of the string which does show the mentioned pop up

we just found it out ourselves , https://github.com/jupyter/nbconvert/issues/754

AnxiousSeal95 , the roaming people, how can I find them actually?

AgitatedDove14 should be, I'll try to create a small example later today or tomorrow

so actually in our case we have an S3 link, but when downloading it, it seems correct.. there is probably some way to make an S3 path open up in the browser by default

AgitatedDove14 , as a temporary workaround, I was wondering how hard would it be to actually use the callback mechanism and manually register the notebook html with something like
Logger.current_logger().report_media("html", "url_html", iteration=iteration, url=" ")?

I mean to get top_tasks