Hello, I’m seeing this issue when launching a machine with the app –
2022-07-05 09:55:49,759 - usage_reporter - INFO - Sending usage report for 60 usage seconds, 1 units 2022-07-05 10:56:40 2022-07-05 09:56:36,826 - clearml.Auto-Scaler - INFO - ignoring unknown worker: 'ip-172-16-30-141:0' 2022-07-05 09:56:36,862 - clearml.Auto-Scaler - INFO - Found 1 tasks in queue 'aws-cpu-clear-managed-autoscale' 2022-07-05 09:56:36,862 - clearml.Auto-Scaler - INFO - Spinning new instance resource='clear_managed_cpu_machines', prefix='clear_machines_pool_4', queue='aws-cpu-clear-managed-autoscale' 2022-07-05 09:56:37,846 - clearml.Auto-Scaler - ERROR - Failed to start new instance (resource 'clear_managed_cpu_machines'), Error: unsupported operand type(s) for *: 'NoneType' and 'int' Traceback (most recent call last): File "/root/.clearml/venvs-builds/3/task_repository/clearml-apps.git/apps/auto_scaler/auto_scaler.py", line 311, in supervisor instance_id = self.driver.spin_up_worker(resource_conf, worker_prefix, queue, task_id=task_id) File "/root/.clearml/venvs-builds/3/task_repository/clearml-apps.git/apps/auto_scaler/cloud_driver.py", line 150, in spin_up_worker instance_id, region = self._spin_up_worker(resource_conf, worker_prefix, queue_name, task_id) File "/root/.clearml/venvs-builds/3/task_repository/clearml-apps.git/apps/auto_scaler/aws_driver.py", line 134, in _spin_up_worker "MaxAttempts": int(resource_conf["regular_instance_rollback_timeout"] * 60 / SPOT_INSTANCE_WAITER_DELAY) + 1 TypeError: unsupported operand type(s) for *: 'NoneType' and 'int' 2022-07-05 09:56:38,344 - clearml.Auto-Scaler - INFO - Idle for 60.00 seconds
It also repeats every ~15 minutes and leaves running machines behind (seemingly launches new machine every time). So I’m left with +/- 105 running unusable cloud machines from yesterday just because they haven’t been shut down by the launcher script 🤯