I think there's a bug with debugging_session, I'll take a look at it
I have managed to connect. Our EC2 instances run in a private subnet so the ssh connection was not working for that reason I believe. Once I connected to my VPN it now worked.
I ran again without the debug mode option and got this error:
>
> Starting Task Execution:
>
>
> Traceback (most recent call last):
> File "/root/.clearml/venvs-builds/3.6/code/interactive_session.py", line 377, in <module>
> from tcp_proxy import TcpProxy
> ModuleNotFoundError: No module named 'tcp_proxy'
>
> Process failed, exit code 1
No particular reason. This was our first time trying it and it seemed the quickest way to get off the ground. When I try without I have a similar error trying to connect although that could be due to the instance.
no, quite the opposite. I think the issue is with using the debugging session option - why did you use that?
(apologies for delay @<1523701087100473344:profile|SuccessfulKoala55> , we got called into meetings. Really appreciate your reactivity!)
I did not run clearml-session from within the git repo.
I did not touch the interactive session code at all.
I installed clearml-session using pip and ran the above command with a task id from a task I'd already run.
@<1546665634195050496:profile|SolidGoose91> commented there again, but just to make this discussion quicker - did you call Task.init() somewhere? Did you modify the interactive session code in any way? (I saw edits in the uncommitted changes)?
It was a debugging session. We haven’t yet tried a “Standard” non-debugging clearml session.
run how? by using a queue monitored by the autoscaler when running the clearml-session command line tool?
We were just trying to run a plain clearml-session on the AWS autoscaler.
Tagging @<1529271085315395584:profile|AmusedCat74> my colleague with whom we ran into this issue.
keep in mind this is not a file that can import other modules since its entire contents (and the contents of other files, tcp_proxy.py for example) is being injected into the session
@<1546665634195050496:profile|SolidGoose91> can you share a way to reproduce that? From what I see this should be working...
I'll try to release an RC as soon as possible
@<1546665634195050496:profile|SolidGoose91> this is actually not trivial (I'll check it out now and explain in the PR)
No problem 🙂 Once you’ve merged it, what do we need to do to get the updated version please?
Thanks for the ping, for some reason I didn't see the email 😞
@<1523701087100473344:profile|SuccessfulKoala55> I think you’ve been tagged in the PR 🙂