Logs from the first task itself:

1721797284959 myDNSName:0 INFO task 20e5aaa6a8df4b1bbdc9fd5ccdcb5e7d pulled from 179eb7123cc04aa7a9a701890ac8ba0b by worker myDNSName:0

1721797290216 myDNSName:0 DEBUG DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): serverAddress:8008
DEBUG:urllib3.connectionpool:

 "GET /auth.login HTTP/1.1" 200 615
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): serverAddress:8008
DEBUG:urllib3.connectionpool:

 "GET /v2.5/tasks.get_by_id HTTP/1.1" 200 1790
DEBUG:urllib3.connectionpool:Resetting dropped connection: serverAddress
DEBUG:urllib3.connectionpool:

 "GET /tasks.dequeue HTTP/1.1" 200 313
DEBUG:urllib3.connectionpool:Resetting dropped connection: serverAddress
DEBUG:urllib3.connectionpool:

 "GET /v2.5/tasks.started HTTP/1.1" 200 331
DEBUG:clearml_agent.session:Run by interpreter: /opt/clearml/.venv/bin/python3.11
Current configuration (clearml_agent v1.8.1, location: /tmp/.clearml_agent.k9o67umy.cfg):
----------------------
api.version = 1.5
api.verify_certificate = true
api.default_version = 1.5
api.http.max_req_size = 15728640
api.http.retries.total = 240
api.http.retries.connect = 240
api.http.retries.read = 240
api.http.retries.redirect = 240
api.http.retries.status = 240
api.http.retries.backoff_factor = 1.0
api.http.retries.backoff_max = 120.0
api.http.wait_on_maintenance_forever = true
api.http.pool_maxsize = 512
api.http.pool_connections = 512
api.auth.token_expiration_threshold_sec = ****
api.api_server = 

api.web_server = 

api.files_server = 

api.credentials.access_key = XTM42TSDNPGYYO3GJDLS
api.credentials.secret_key = ****
api.host = 

agent.worker_id = myDNSName:0
agent.worker_name = myDNSName
agent.force_git_ssh_protocol = false
agent.python_binary =
agent.package_manager.type = poetry
agent.package_manager.pip_version =
agent.package_manager.system_site_packages = false
agent.package_manager.force_upgrade = false
agent.package_manager.conda_channels.0 = pytorch
agent.package_manager.conda_channels.1 = conda-forge
agent.package_manager.conda_channels.2 = nvidia
agent.package_manager.conda_channels.3 = defaults
agent.package_manager.priority_optional_packages.0 = pygobject
agent.package_manager.torch_nightly = false
agent.package_manager.poetry_files_from_repo_working_dir = false
agent.venvs_dir = /home/clearml_user/.clearml/venvs-builds
agent.venvs_cache.max_entries = 10
agent.venvs_cache.free_space_threshold_gb = 2.0
agent.venvs_cache.path = ~/.clearml/venvs-cache
agent.vcs_cache.enabled = true
agent.vcs_cache.path = /home/clearml_user/.clearml/vcs-cache
agent.venv_update.enabled = false
agent.pip_download_cache.enabled = true
agent.pip_download_cache.path = /home/clearml_user/.clearml/pip-download-cache
agent.translate_ssh = true
agent.reload_config = false
agent.docker_pip_cache = /home/clearml_user/.clearml/pip-cache
agent.docker_apt_cache = /home/clearml_user/.clearml/apt-cache
agent.docker_force_pull = false
agent.default_docker.image = python:3.11.9-slim-bookworm
agent.enable_task_env = false
agent.sanitize_config_printout = ****
agent.hide_docker_command_env_vars.enabled = true
agent.hide_docker_command_env_vars.parse_embedded_urls = true
agent.abort_callback_max_timeout = 1800
agent.docker_internal_mounts.sdk_cache = /clearml_agent_cache
agent.docker_internal_mounts.apt_cache = /var/cache/apt/archives
agent.docker_internal_mounts.ssh_folder = ~/.ssh
agent.docker_internal_mounts.ssh_ro_folder = /.ssh
agent.docker_internal_mounts.pip_cache = /root/.cache/pip
agent.docker_internal_mounts.poetry_cache = /root/.cache/pypoetry
agent.docker_internal_mounts.vcs_cache = /root/.clearml/vcs-cache
agent.docker_internal_mounts.venv_build = ~/.clearml/venvs-builds
agent.docker_internal_mounts.pip_download = /root/.clearml/pip-download-cache
agent.apply_environment = true
agent.apply_files = true
agent.custom_build_script =
agent.disable_task_docker_override = false
agent.git_user =
agent.git_pass = ****
agent.force_git_root_python_path = true
agent.debug = true
agent.default_python = 3.11
agent.cuda_version = 0
agent.cudnn_version = 0
sdk.storage.cache.default_base_dir = ~/.clearml/cache
sdk.storage.cache.size.min_free_bytes = 10GB
sdk.storage.direct_access.0.url = file://*
sdk.metrics.file_history_size = 100
sdk.metrics.matplotlib_untitled_history_size = 100
sdk.metrics.images.format = JPEG
sdk.metrics.images.quality = 87
sdk.metrics.images.subsampling = 0
sdk.metrics.tensorboard_single_series_per_graph = false
sdk.network.metrics.file_upload_threads = 4
sdk.network.metrics.file_upload_starvation_warning_sec = 120
sdk.network.iteration.max_retries_on_server_error = 5
sdk.network.iteration.retry_backoff_factor_sec = 10
sdk.aws.s3.key =
sdk.aws.s3.secret = ****
sdk.aws.s3.region =
sdk.aws.boto3.pool_connections = 512
sdk.aws.boto3.max_multipart_concurrency = 16
sdk.log.null_log_propagate = false
sdk.log.task_log_buffer_capacity = 66
sdk.log.disable_urllib3_info = true
sdk.development.task_reuse_time_window_in_hours = 72.0
sdk.development.vcs_repo_detect_async = true
sdk.development.store_uncommitted_code_diff = true
sdk.development.support_stopping = true
sdk.development.default_output_uri =
sdk.development.force_analyze_entire_repo = false
sdk.development.suppress_update_message = false
sdk.development.detect_with_pip_freeze = false
sdk.development.worker.report_period_sec = 2
sdk.development.worker.ping_period_sec = 30
sdk.development.worker.log_stdout = true
sdk.development.worker.report_global_mem_used = false

DEBUG:urllib3.connectionpool:Resetting dropped connection: serverAddress
DEBUG:urllib3.connectionpool:

 "GET /v2.5/tasks.started HTTP/1.1" 200 355
Executing task id [20e5aaa6a8df4b1bbdc9fd5ccdcb5e7d]:
repository =
branch =
version_num =
tag =
docker_cmd =
entry_point = foo.py
working_dir = .

DEBUG:clearml_agent.commands.worker:Searching for python3.11
DEBUG:clearml_agent.commands.worker:Found: python3.11
created virtual environment CPython3.11.2.final.0-64 in 447ms
  creator CPython3Posix(dest=/home/clearml_user/.clearml/venvs-builds/3.11, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/clearml_user/.local/share/virtualenv)
    added seed packages: pip==24.1.2, setuptools==70.1.1, wheel==0.43.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator


INFO:clearml_agent.commands.worker:found literal script in `script.diff`
DEBUG:clearml_agent.commands.worker:selected execution directory: /home/clearml_user/.clearml/venvs-builds/3.11/code


Executing: (PosixPath('/home/clearml_user/.clearml/venvs-builds/3.11/bin/python'), '-m', 'pip', '--disable-pip-version-check', 'install', 'pip', '--upgrade')

1721797295107 myDNSName:0 DEBUG Looking in indexes: 

, 

Requirement already satisfied: pip in /home/clearml_user/.clearml/venvs-builds/3.11/lib/python3.11/site-packages (24.1.2)

1721797300154 myDNSName:0 DEBUG Executing: (PosixPath('/home/clearml_user/.clearml/venvs-builds/3.11/bin/python'), '-m', 'pip', '--disable-pip-version-check', 'list')
Executing: (PosixPath('/home/clearml_user/.clearml/venvs-builds/3.11/bin/python'), '-m', 'pip', '--disable-pip-version-check', 'install', 'Cython')
Looking in indexes: 

, 

Collecting Cython
  Using cached Cython-3.0.10-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.metadata (3.2 kB)
Using cached Cython-3.0.10-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (3.4 MB)
Installing collected packages: Cython

1721797305206 myDNSName:0 DEBUG Process failed, exit code 1
1721797305257 myDNSName:0 DEBUG Successfully installed Cython-3.0.10
INFO:clearml_agent.commands.worker:Found task requirements section, trying to install
Executing: (PosixPath('/home/clearml_user/.clearml/venvs-builds/3.11/bin/python'), '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/tmp/cached-reqswqel5t8m.txt')
Looking in indexes: 

, 

ERROR: Could not find a version that satisfies the requirement clearml_showcase==0.1 (from versions: none)
ERROR: No matching distribution found for clearml_showcase==0.1
INFO:clearml_agent.commands.worker:Traceback (most recent call last):
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/commands/worker.py", line 3221, in install_requirements_for_package_api
    package_api.load_requirements(cached_requirements)
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/package/pip_api/venv.py", line 41, in load_requirements
    super(VirtualenvPip, self).load_requirements(requirements)
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/package/pip_api/system.py", line 61, in load_requirements
    self.install_from_file(path)
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/package/pip_api/system.py", line 35, in install_from_file
    self.run_with_env(('install', '-r', path) + self.install_flags(), cwd=self.cwd)
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/package/pip_api/system.py", line 94, in run_with_env
    return (command.get_output if output else command.check_call)(stdin=DEVNULL, env=env, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/process.py", line 198, in check_call
    return self.call_subprocess(subprocess.check_call, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/clearml/.venv/lib/python3.11/site-packages/clearml_agent/helper/process.py", line 245, in call_subprocess
    return func(list(self), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/clearml_user/.clearml/venvs-builds/3.11/bin/python', '-m', 'pip', '--disable-pip-version-check', 'install', '-r', '/tmp/cached-reqswqel5t8m.txt']' returned non-zero exit status 1.