Update: I see that by default it uses 10022 as the remote SSH port, so I’ve opened that as well (still getting the “tunneling failed” message though).

I’ve also noticed this log in the agent machine:
2021-07-09 05:38:37,766 - clearml - WARNING - Could not retrieve remote configuration named 'SSH' Using default configuration: {'ssh_host_ecdsa_key': '-----BEGIN EC PRIVATE KEY-----\{private key here}