Another option is to pull Tasks from a dedicated queue and use the LocalClearMLJob to spwan themThis sounds like it can work. we are talking about something like:
` #<Machine 1>
#Init Optimizer with some dedicated queue
<Machine 2>
heavy one time Common Initialization
while True:
# sample queue
# enqueue with LocalClearMLJob
# Execute Something
# report results
<Machine i>
heavy one time Common Initialization
while True:
# sample same queue
# enqueue wi...

Thanks Martin! I'll test it in the following days, I'll keep you updated!

it works.. but I am actually trying to access the status not the stats, is that an option?

the offline error is different

But it does make me think, if instead of changing the optimizer I launch a few workers that "pull" enqueued tasks, and then report values for them in such a way that the optimizer is triggered to collect the results? would it be possible?

I am getting the following error when I try the above:
` /opt/conda/envs/torch_38/lib/python3.8/site-packages/clearml/backend_api/session/client/client.py in new_func(self, *args, **kwargs)
374 @wrap
375 def new_func(self, *args, **kwargs):
--> 376 return Response(self.session.send(request_cls(*args, **kwargs)))
377
378 new_func.name = new_func.qualname = action

TypeError: init() missing 1 required positional argument: 'items' `

no programmatic python options? could be nice..

I am trying to mimic an agent pulling a task, and while running it syncing some custom configuration dict I have according to the task configuration (overriding the defaults)

SuccessfulKoala55 , thanks I was looking for a way to do it programatically.. solved now, thanks
request = url+ '/v2.14/debug.ping' r = requests.post(request) serverIsResponsive = r.json()['meta']['result_code'] == 200

is there a fundamental reason why is this only enabled in --docker mode?

cool thanks! local is quite confusing in this context.. but works 🙂

AgitatedDove14 , what I meant by manually filtering, at the moment, to combine the information of metric values + HP point, I pull all the parameters, and then manually filter on the HP keys (manually=I have to plug them in, they are not part of optimizer object)

The "notebook preview" link doesn't pop-up the s3 credentials, however the "notebook" artifact when clicking on the download button does show this pop up
they actually seem to be different in the way that the UI server is showing them.. one is a link (the html) and has no pop up, the other one (.ipynb) is a text with a download button in the end of the string which does show the mentioned pop up

does my explanation makes sense?

I understand that to report any value should be presented as string, how does the "inverse casting" work when I pull some value from the config?

AgitatedDove14 , TimelyPenguin76 , a small blast from the past
Unfortunately it seems like this is not working for backslash escape character
https://demoapp.demo.clear.ml/projects/7eaa1749475d4ad4bd21a5456fd2e157/experiments/3efe981238e543c8b6ad682dd13c72bc/output/hyper-params/hyper-param/General
https://colab.research.google.com/drive/1w5lQGxsblnLGlhJEDH_b0aIiUvjGeLjy?usp=sharing

this one is with the brave browser but I get the same with chrome

It would be very very useful for my use case, and I believe a relatively popular use case in general for example when using regular expression configurations

🙏 thanks!

is there an available reference for how I can use the API calls with the python API? it is not clear to me from what you shared

SuccessfulKoala55
` import plotly.express as px
df = px.data.gapminder()
fig = px.scatter(df, x="gdpPercap", y="lifeExp", animation_frame="year", animation_group="country",
size="pop", color="continent", hover_name="country",
log_x=True, size_max=55, range_x=[100, 100000], range_y=[25, 90])

task.get_logger().report_plotly(title="TEST", series="sepal", iteration=0, figure=fig) `Thanks

AgitatedDove14 , the issue you mention does not relate to this discussion

Lets say I inherit from the Optimizer (you mean HyperParameterOptimizer class? or SearchStrategy?), implement a custom logic for experiment creation logic,
what does it actually exposes? creating an experiment means defining a task, enqueue it and then? I am trying to think what you meant I can put in the logic such that I get the desired effect

you can see the behavior here:

thanks AgitatedDove14 ! this is what I was looking for

in the same queue I have about 60 pending, and I want the bottom 30 to be the top 30

changing the queue order cool, but a bit too limited.. I have 30 jobs I want to multi select and push up to first priority.. this is a lot of manual labor..

Thanks AgitatedDove14 , I actually used it before and forgot it exists

from the example I shared above
config_files/cfg.py
` from hydra.core.config_store import ConfigStore
from dataclasses import dataclass

@dataclass
class MasterConfig:
test: str = 'test'

cs = ConfigStore.instance()
cs.store(name="config", node=MasterConfig) `

imagine the beautiful PR showing such a feature 👀