This one - https://github.com/allegroai/clearml/blob/master/examples/optimization/hyper-parameter-optimization/hyper_parameter_optimizer.py

Ah ok there’s only optimizer.stop in the example

just that the task itself is still Running state

no task.close()

Optuna

Ok, just my ignorance then? 🙂

The "Optimizer task" will continue to run as long as there are sub-Tasks it launched.
Is anything else running/pending ?

The Optimizer task is taking a lot of time to complete. Is it doing something here:

It completed after the max_job limit (10)

Yep this is optuna "testing the water"

I am running from noebook and cell has returned

Well the Task will close when you shut down the notebook 🙂

Yeah, Curious - is a lot of clearml usecases not geared for notebooks?

I am running from noebook and cell has returned

how do you see things being used as the most normal way?

Tried context provider for Task?

I guess that would only make sense inside notebooks ?!

Something like:

with Task() as t: #train

Tried context provider for Task?

The job itself doesn’t have any other param

Only one param, just playing around

So this is optuna 🙂 the idea is it will test which parameters have potential (with early stopping), then launch a subset of the selected parameters

You mean the job with the exact same arguments ?

Yes

You mean the job with the exact same arguments ?
do you have other arguments you are passing ?
Are you using Optuna / HBOB ?

Notice Optuna will do TPE & hyper band Bayesian optimization to find the best combination

Ok, just my ignorance then? 

LOL, no it is just that with a single discrete parameter the strategy makes less sense 🙂

It completed after the max_job limit (10)

Yeah, Curious - is a lot of clearml usecases not geared for notebooks?

That is somewhat correct, notebooks are not actually used with a lot of deep-learning projects as they require entire repository to support.
I guess generally speaking the workflow is, "test your code" (i.e. small scale with limited data), then clone and enqueue for remote execution.
That said, I think it will be great to expand the support.
TrickySheep9 I like the idea of context for Tasks, can you expand on how this is used as part of a (jupyter) workflow ?

No, all of them completed!