Unanswered
Hi All, I Am Trying To Execute Somewhat Custom Hpo Scheme With Clearml.
I Would Want That A Single Running Python Script Will Be Able To Sample The Optimizer, Init A Task And Report The Result Multiple Times. I Didn'T Find Anything Similar In The Docs Or
This might work (I have to admit I haven't had the time to test, please let me know if it works, so we could push it as a cool new feature 🙂 )
` class LocalClearmlJob(ClearmlJob):
def init(self, *args, **kwargs):
super(LocalClearmlJob, self).init(*args, **kwargs)
def launch(self, queue_name=None):
# type: (str) -> bool
if self._is_cached_task:
return False
# create the subprocess
cmd = self.task.data.execution.script.entrypoint
python = sys.executable
env = dict(**os.environ)
env['CLEARML_TASK_ID'] = env['TRAINS_TASK_ID'] = self.task.id
env['CLEARML_LOG_TASK_TO_BACKEND'] = 1
env['CLEARML_SIMULATE_REMOTE_TASK'] = 1
p = subprocess.Popen(args=[python, cmd], cwd=os.getcwd(), env=env)
return True
an_optimizer = HyperParameterOptimizer(max_number_of_concurrent_tasks=1, ...)
an_optimizer.set_default_job_class(LocalClearmlJob)
an_optimizer.start()
an_optimizer.wait()
an_optimizer.stop() `
This will code will spin a subprocess running the original Task as if it is running by the agent, only locally.
This means the optimizer can control the parameters, and you are running all jobs locally.
wdyt?
150 Views
0
Answers
3 years ago
one year ago