Hi, trying to debug clearml-session on an azure instance. Cannot seem to connect to it.

HELLO FELLOW AZURE USER FROM THE FUTURE: A SOLUTION WAS FOUND: CREATE TCP RULE FOR PORT 10022 INBOUND

Right now I’ve boiled it down to the basics to try and find out what’s going on.

My command is
clearml-session --vscode-server false --jupyter-lab false --queue default --public-ip
tried with and without public-ip, stuck in permanent:
SSH tunneling failed, retrying in 3 seconds Starting SSH tunnelI think the problem is in the initial setup of the session task. Here is the output to the DevOps task console:
ClearML results page: `
2022-06-12 08:16:12,547 - clearml - WARNING - Could not retrieve remote configuration named 'SSH'
Using default configuration: {'ssh_host_ecdsa_key': '-----BEGIN EC PRIVATE KEY-----*\n-----END EC PRIVATE KEY-----\n', 'ssh_host_ed25519_key': '-----BEGIN OPENSSH PRIVATE KEY-----**\n-----END RSA PRIVATE KEY-----\n', 'ssh_host_rsa_key__pub': 'ssh-rsa ****', 'ssh_host_ecdsa_key__pub': 'ecdsa-sha2-nistp256 ****', 'ssh_host_ed25519_key__pub': None}
Applying vault configuration
Installing SSH Server on Merlin-dev [10.2.0.11]
Unable to load host key "/datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_rsa_key.pub": invalid format
Unable to load host key: /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_rsa_key.pub
Unable to load host key "/datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ecdsa_key.pub": invalid format
Unable to load host key: /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ecdsa_key.pub
Unable to load host key: /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ed25519_key.pub

SSH Server running on Merlin-dev [10.2.0.11] port 10022

LOGIN u:root p:None

no jupyterlab to monitor - going to sleep This is the sshd_config created PermitRootLogin yes
ClientAliveInterval 10
ClientAliveCountMax 20
AllowTcpForwarding yes
UsePAM yes
AuthorizedKeysFile /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/authorized_keys
PidFile /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/sshd.pid
AcceptEnv CLEARML_API_ACCESS_KEY CLEARML_API_SECRET_KEY CLEARML_API_ACCESS_KEY CLEARML_API_SECRET_KEY
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ecdsa_key
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ed25519_key
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_rsa_key
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_rsa_key.pub
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ecdsa_key.pub
HostKey /datadrive/clearml_cache/venvs-builds/3.8/code/.clearml_session_sshd/ssh_host_ed25519_key.pub `