I think we need logging here: https://github.com/allegroai/clearml-session/blob/bf1851cd3831c19cc0eadd9b2ffc0613f97f16e1/clearml_session/main.py#L564

to know why exactly ssh tunneling fails

yes I can.

Many thanks FiercePenguin76 !

Oh in that case add --remote-gateway <external_ip> It will connect to the provided address instead of the local one. (you can also add --public-ip which will automatically resolve the public IP of the server

it is missing in CLI, but I was able to set external_ssh_port and external_address in GUI. It was certainly a step forward, but still failed

I guess that’s because ngrok is not like a Dynamic DNS

well, I first run clearml-session to start everything on the remote machine, then I close the local process (while Interactive is still running on the remote machine)

I use ngrok to connect to it

` Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '<CENSORED>' (ECDSA) to the list of known hosts.
Enter passphrase for key '/Users/jevgenimartjushev/.ssh/id_rsa': <CENSORED>

SSH tunneling failed, retrying in 3 seconds `

Then I ssh into the remote machine using ngrok hostname and tunnel the port for Jupyter

Hmmm, yes we should definitely add --debug (if you can, please add a GitHub issue so it is not forgotten).
FiercePenguin76 Specifically are you able to ssh manually to <external_address>:<external_ssh_port> ?

I think the clearml-session CLI is missing the ability to add cutom port to the external address, does that make sense ?

ngrok to connect to the remote server at the office?
That makes sense, I guess this is the equivalent of using a VPN, from that point onward clearml-session can directly access the remote machine, right?

I tried both, didn’t work

Hi FiercePenguin76

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

You are correct, clearml-session assumes it can SSH into the remote agent machine, from that point it automatically tunnels all other connections on top of the original SSH (well with some fancy SSH keep-alive proxy).
I'm assuming that from home you cannot connect to the SSH machine at the office, which makes sense, but out of curiosity how did you manage to SSH to it manually ?

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

https://github.com/allegroai/clearml-session/issues/1

For others, who haven’t heard about ngrok:
Ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.

So I thought, maybe I can tell clearml-session to use hostname from ngrok