Hi I upgraded some of my k8s glue to use the latest clearml-agent 1.1.0 and receive this error when the agent pulls a task.
Pulling task xxxbbbxxxbb launching on kubernetes cluster
Pushing task xxxbbbxxxbb into temporary pending queue
Kubernetes scheduling task id=xxxbbbxxxbb
FATAL ERROR:
Traceback
file "/usr/local/lib/python3.6/dist-packages/clearml-agent/commands/worker.py, line 1509, in Daemon gpu-queues=dynamic_gpus
..
..
file "/usr/local/lib/python3.6/dist-packages/clearml-agent/glue/k8s.py in run_one_task line 505
Print("kubectl output:\n...
Unicodeencodeerror:'ascii' codec can't encode characters in position 296-297: ordinal not in range (128)
Hi SubstantialElk6
Unicodeencodeerror:'ascii' codec can't encode characters in position 296-297: ordinal not in range (128) (edited)
I'm assuming this is the usual UTF8 missing from the container.
Can you try to launch it with PYTHONIOENCODING=utf-8 ?
Well, it probably crashed due to this error when trying to print the kubectl output - any reason the output isn't ascii?
It also stopped taking in tasks from the queue after that.
