Thanks! 🙂

AgitatedDove14 suppose that we are doing some optimization task (parameter search). This is a task where generally we want to minimize some metric m , but it will be enough to have, say 3 occurences when m<THRESHOLD and when it will happen, we stop the search (and free the resources, that can be needed for some further step)

SuccessfulKoala55 thank you for the response; what about the second part of question (stopping)?

AgitatedDove14 do you know if it possible not to open ports on machines B_i where agents reside?

The use case was that server with repo wasn't responding for a while and I was thinking how to solve that. Thanks for the answer!

Commits, that are not pushed to the repo

Can I do this to specify which worker should execute that task?
CLEARML_WORKER_NAME=<worker_name> clearml-agent execute --id <task_id>

How to do that?

AgitatedDove14 shouldn't it be
while not an_optimizer.wait(timeout=1.0):instead of
while an_optimizer.wait(timeout=1.0):in the first code block?

AgitatedDove14 in fact in our case we want to use simple strategies, RandomSearch is enough, but the problem is that we need to change the ranges dynamically

https://github.com/allegroai/clearml/issues/864

hmm, is there a way to name the workers in some other way?

clearml_agent: ERROR: Instance with the same WORKER_ID [our_machine:gpu0] is already running

or at least I can't specify such

Hmm, it is hard to specify the way

ClearML Server Version: 1.7.0-232

Regarding this last question - I know that there is possibility to set up some budget - for example seconds of running after which optimization stops. But is there a possibility to specify a boolean condition when work should stop?

Because it has no coincidence with some specific actions

I am using UI and I am clicking select all. If it is calling API server then yes

The problem is that we have a a complex configuration of pipeline. Configuration changes quite frequently and we would not like to run the pipeline every time configuration changes, but we would like to have it scheduled in some defined periods.

Do you have an idea of some workaround / alternative solution for that problem?

because when I run that normally it differentiates workers basing on gpu that it is using

Wow, that looks insteresting 🙂 Thank you, AgitatedDove14

SuccessfulKoala55 Thank you for the response! Let me elaborate a bit to check if I understand this correctly.
We have a time-consuming task T based on optimization for parameters. We want to run hyperparameter optimization for T, suppose that we want to run it for 100 sets of parameters.
We want to leverage the fact that we have n machines to make the work parallel.

So for that we use https://clear.ml/docs/latest/docs/references/sdk/hpo_optimization_hyperparameteroptimizer/ , we run Agent...

From UI: right click, abort

CostlyOstrich36 have you ever seen something like my case maybe?

AgitatedDove14 one more question regarding this issue
Is it possible to change parameter space dynamically.
(dummy) example:
Our optimization is a task when we sample from [1,2,3] twice. At the situation when 3 is chosen twice, eliminate 3 from one sampling range, so make the sampling x1 from [1,2,3] and x2 from [1,2]

WARNING: You are using pip version 20.1.1; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip'
command.
Retrying (Retry(total=239, connect=239, read=240, redirect=240, status=240)) after co
nnection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at
` 0x7faf9da78400>: Failed to establish a ...

No. I would like to use TaskScheduler for pipelines. For now it seems to me, that I need to firstly run whole pipeline to get it's id.

I would like to define the pipeline but not run it before it is run by the scheduler

Ok, thanks!

no, it is everything on my local machine