Hmmm, yes we should definitely add --debug (if you can, please add a GitHub issue so it is not forgotten).
FiercePenguin76 Specifically are you able to ssh manually to <external_address>:<external_ssh_port> ?
I think we need logging here: https://github.com/allegroai/clearml-session/blob/bf1851cd3831c19cc0eadd9b2ffc0613f97f16e1/clearml_session/main.py#L564
` Remote machine is ready
Setting up connection to remote session
Starting SSH tunnel
Warning: Permanently added '<CENSORED>' (ECDSA) to the list of known hosts.
Enter passphrase for key '/Users/jevgenimartjushev/.ssh/id_rsa': <CENSORED>
SSH tunneling failed, retrying in 3 seconds `
it is missing in CLI, but I was able to set external_ssh_port and external_address in GUI. It was certainly a step forward, but still failed
I think the clearml-session CLI is missing the ability to add cutom port to the external address, does that make sense ?
I guess that’s because ngrok is not like a Dynamic DNS
Oh in that case add --remote-gateway <external_ip> It will connect to the provided address instead of the local one. (you can also add --public-ip which will automatically resolve the public IP of the server
For others, who haven’t heard about ngrok:Ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels.
Then I ssh into the remote machine using ngrok hostname and tunnel the port for Jupyter
well, I first run clearml-session to start everything on the remote machine, then I close the local process (while Interactive is still running on the remote machine)
ngrok to connect to the remote server at the office?
That makes sense, I guess this is the equivalent of using a VPN, from that point onward clearml-session can directly access the remote machine, right?
So I thought, maybe I can tell clearml-session to use hostname from ngrok
Hi FiercePenguin76
Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.
You are correct, clearml-session assumes it can SSH into the remote agent machine, from that point it automatically tunnels all other connections on top of the original SSH (well with some fancy SSH keep-alive proxy).
I'm assuming that from home you cannot connect to the SSH machine at the office, which makes sense, but out of curiosity how did you manage to SSH to it manually ?
Here’s my workaround - ignore the fail messages, and manually create an SSH connection to the server with Jupyter port forwarded.