So created queue ... now im getting
python scripts/hyper_parameter_optimizer.py
ClearML Task: created new task id=2ba36b52e6004514a41dc52bb2ba169c
ClearML results page:
[I 2024-08-18 13:31:41,133] A new study created in memory with name: 2ba36b52e6004514a41dc52bb2ba169c
Progress report #0 completed, sleeping for 0.25 minutes
2024-08-18 13:31:41,370 - clearml.automation.optimization - INFO - Creating new Task: {'General/layer_1': 256, 'General/layer_2': 384, 'General/batch_size': 160, 'General/epochs': 30}
2024-08-18 13:31:41,627 - clearml.automation.optimization - INFO - Creating new Task: {'General/layer_1': 512, 'General/layer_2': 384, 'General/batch_size': 128, 'General/epochs': 30}
Progress report #1 completed, sleeping for 0.2 minutes
Progress report #2 completed, sleeping for 0.2 minutes
Progress report #3 completed, sleeping for 0.2 minutes
Progress report #4 completed, sleeping for 0.2 minutes
Progress report #5 completed, sleeping for 0.2 minutes
Progress report #6 completed, sleeping for 0.2 minutes
Progress report #7 completed, sleeping for 0.2 minutes
Progress report #8 completed, sleeping for 0.2 minutes
Progress report #9 completed, sleeping for 0.2 minutes
Progress report #10 completed, sleeping for 0.2 minutes
Progress report #11 completed, sleeping for 0.2 minutes
Progress report #12 completed, sleeping for 0.2 minutes
[W 2024-08-18 13:34:14,978] Trial 0 failed with parameters: {'General/layer_1': 256, 'General/layer_2': 384, 'General/batch_size': 160, 'General/epochs': 30} because of the following error: TypeError("'NoneType' object is not subscriptable").
Traceback (most recent call last):
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/optuna/study/_optimize.py", line 196, in _run_trial
value_or_values = func(trial)
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/clearml/automation/optuna/optuna.py", line 92, in objective
objective_metric = objective_metric[0]
TypeError: 'NoneType' object is not subscriptable
[W 2024-08-18 13:34:14,980] Trial 0 failed with value None.
Progress report #13 completed, sleeping for 0.2 minutes
Progress report #14 completed, sleeping for 0.2 minutes
Progress report #15 completed, sleeping for 0.2 minutes
[W 2024-08-18 13:34:56,087] Trial 1 failed with parameters: {'General/layer_1': 512, 'General/layer_2': 384, 'General/batch_size': 128, 'General/epochs': 30} because of the following error: TypeError("'NoneType' object is not subscriptable").
Traceback (most recent call last):
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/optuna/study/_optimize.py", line 196, in _run_trial
value_or_values = func(trial)
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/clearml/automation/optuna/optuna.py", line 92, in objective
objective_metric = objective_metric[0]
TypeError: 'NoneType' object is not subscriptable
[W 2024-08-18 13:34:56,088] Trial 1 failed with value None.
Exception in thread Thread-5 (_daemon):
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
....
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/optuna/study/_optimize.py", line 247, in _run_trial
raise func_err
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/optuna/study/_optimize.py", line 196, in _run_trial
value_or_values = func(trial)
File "/home/razoffice/corsound_vfm/venv_raz/lib/python3.10/site-packages/clearml/automation/optuna/optuna.py", line 92, in objective
objective_metric = objective_metric[0]
TypeError: 'NoneType' object is not subscriptable
['333ed1cf667c4ba390844b757a2625ef', '04921bcae42c403f99de46f1c584372d']
`Study.stop` is supposed to be invoked inside an objective function or a callback.
We are done, good bye
I didnt modified your scripts , only the execution_queue variable
I suggest you review ClearML onboarding videos on the Youtube channel and read about the basics of the agent/orchestration.
You also need an agent listening to the queue you're enqueuing to
Please see the error:
2024-08-18 12:55:25,030 - clearml.automation.job - WARNING - Error enqueuing Task <clearml.task.Task object at 0x723c45320610> to 1xGPU: Could not find queue named "1xGPU"
You don't have a queue called 1xGPU.
Sure i did it but im getting the following
A new study created in memory with name: 8ea132f808934f1a962aeb89cff31bd0
Progress report #0 completed, sleeping for 0.25 minutes
2024-08-18 12:55:24,959 - clearml.automation.optimization - INFO - Creating new Task: {'General/layer_1': 128, 'General/layer_2': 256, 'General/batch_size': 160, 'General/epochs': 30}
2024-08-18 12:55:25,030 - clearml.automation.job - WARNING - Error enqueuing Task <clearml.task.Task object at 0x723c45320610> to 1xGPU: Could not find queue named "1xGPU"
[W 2024-08-18 12:55:25,030] Trial 0 failed with parameters: {'General/layer_1': 128, 'General/layer_2': 256, 'General/batch_size': 160, 'General/epochs': 30} because of the following error: The value None could not be cast to float..
[W 2024-08-18 12:55:25,031] Trial 0 failed with value None.
2024-08-18 12:55:25,287 - clearml.automation.optimization - INFO - Creating new Task: {'General/layer_1': 384, 'General/layer_2': 128, 'General/batch_size': 96, 'General/epochs': 30} ...
[W 2024-08-18 12:55:26,702] Trial 9 failed with value None.
['0effcb51e242499f9abb0f7c8d6eac60', '067dd618ef4c4db4adeb96553d7e1d44', '809ed889e43c4720b5d3d78883b69fea']
We are done, good bye
You need to run the base first and then the optimizer. The optimizer will clone the base and enqueue it. Read further here - None
@<1523701070390366208:profile|CostlyOstrich36> Hi
I tried this example but not seems im doing it right ....
How does this two scripts communicating between themselves ?
base_template_keras_simple.py
hyper_parameter_optimizer.py
im not understand which is should run first ?
I suggest reviewing what other examples are there
Hi @<1734744942921650176:profile|RipeBee29> , you can find an example here:
None