
Reputation
Badges 1
3 × Eureka!no, I have only one autoscaler and few queues. Maybe it was queue on clear.ml side to restart autoscalers?
Hi,
It looks like the same issue is happening. It seems to be caused by the recent update of the clearml-agent
package to version 2.0.0 .
When I start the queue locally, the agent appears in the list but doesn't pick up any tasks. On the agent side, I get the following error:
FATAL ERROR:
Traceback (most recent call last):
File "***.venv/lib/python3.12/site-packages/clearml_agent/commands/worker.py", line 2128, in daemon
self.run_tasks_loop(
File "***.venv/lib/python3.12/site-pa...
It looks like the task.execute_remotely()
method is somehow broken. Previously, when I used it, the task would run in the queue with the same parameters I set locally. But now the parameters are not being passed correctly, and I end up with two tasks: one that I launched locally (but it ends up running remotely), and another one — but without any parameters at all.
It's strange — how could the agent update have affected this?
Is there a way to override the version of clearml-agent
that gets installed on the worker?
No. I have only one autoscaler. But after 6 hours it's now Running.