that machine will be able to pull and report multiple trials without restarting
What do you mean by "pull and report multiple trials" ? Spawn multiple processes with different parameters ?
If this is the case: the internals of the optimizer could be synced to the Task so you can access them, but this is basically the internal representation, which is optimizer dependent, which one did you have in mind?
Another option is to pull Tasks from a dedicated queue and use the LocalClearMLJob to spwan them
(think another script in the same repository, just launching them, then the script is the Task we enqueue, this is actually an agent inside an agent).
Now going back to the initial problem we are trying to solve:
... without restarting
How long are those trial that restarting becomes a bottle neck ?
(Notice that git repo is cached, python packages are cached, and I would also recommended turning on full venv cache, this ends up in about 10 sec to spin a Task, not very long, I think...)
https://github.com/allegroai/clearml-agent/blob/351f0657c3dcf707659875d7e0a52fa387709978/docs/clearml.conf#L104