Hmmm, yes we should definitely add --debug (if you can, please add a GitHub issue so it is not forgotten).
FiercePenguin76 Specifically are you able to ssh manually to <external_address>:<external_ssh_port> ?

Then I ssh into the remote machine using ngrok hostname and tunnel the port for Jupyter

Many thanks FiercePenguin76 !

Hi FiercePenguin76

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

You are correct, clearml-session assumes it can SSH into the remote agent machine, from that point it automatically tunnels all other connections on top of the original SSH (well with some fancy SSH keep-alive proxy).
I'm assuming that from home you cannot connect to the SSH machine at the office, which makes sense, but out of curiosity how did you manage to SSH to it manually ?

well, I first run clearml-session to start everything on the remote machine, then I close the local process (while Interactive is still running on the remote machine)

` Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '<CENSORED>' (ECDSA) to the list of known hosts.
Enter passphrase for key '/Users/jevgenimartjushev/.ssh/id_rsa': <CENSORED>

SSH tunneling failed, retrying in 3 seconds `

So I thought, maybe I can tell clearml-session to use hostname from ngrok

I think we need logging here: https://github.com/allegroai/clearml-session/blob/bf1851cd3831c19cc0eadd9b2ffc0613f97f16e1/clearml_session/main.py#L564

I guess that’s because ngrok is not like a Dynamic DNS

I tried both, didn’t work

ngrok to connect to the remote server at the office?
That makes sense, I guess this is the equivalent of using a VPN, from that point onward clearml-session can directly access the remote machine, right?

Oh in that case add --remote-gateway <external_ip> It will connect to the provided address instead of the local one. (you can also add --public-ip which will automatically resolve the public IP of the server

For others, who haven’t heard about ngrok:
Ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.

it is missing in CLI, but I was able to set external_ssh_port and external_address in GUI. It was certainly a step forward, but still failed

yes I can.

Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.

I use ngrok to connect to it

https://github.com/allegroai/clearml-session/issues/1

to know why exactly ssh tunneling fails

I think the clearml-session CLI is missing the ability to add cutom port to the external address, does that make sense ?