and is there any way to capture hydra from a notebook as a Configuration? you don't use the typical @hydra.main() but rather call the compose API , and so far in my testing that doesn't capture the OmegaConf in ClearML

yep! None

I will once I figure out the fix!

but maybe that doesn't matter, actually - it might be one session per host I guess

environ{'PYTHONNOUSERSITE': '0',
        'HOSTNAME': 'gfp-science-ml-t3-medium-d579233e8c4b53bc5ad626f2b385',
        'AWS_CONTAINER_CREDENTIALS_RELATIVE_URI': '/_sagemaker-instance-credentials/xxx',
        'JUPYTER_PATH': '/usr/share/jupyter/',
        'SAGEMAKER_LOG_FILE': '/var/log/studio/kernel_gateway.log',
        'PATH': '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tmp/miniconda3/condabin:/tmp/anaconda3/condabin:/tmp/miniconda2/condabin:/tmp/anaconda2/condabin'...

still empty

so my reading of the jupyter-kernel-gateway docs is that each session is containerized, so each notebook "session" is totally isolated

Yeah, I think that'd work

if I instead change the request url to f"http://{server_info['hostname']}:{server_info['port']}/api/sessions" then it gets a 200 response... however , the response is an empty list

but r.json() is an empty list

that fails

but one possible workaround is to try to figure out if it's running in a gateway and then find the only notebook running on that server

if I use the same kernel there'll be two

sadly no

and the only calls to "uri": "/api/sessions" are the ones I made during testing - sagemaker doesn't seem to ever call that itself

and cat /var/log/studio/kernel_gateway.log | grep ipynb comes up empty

at least in 2018 it returned sessions! None

sounds good!

I can get it to run up to here: None

poking around a little bit, and clearml.backend_interface.task.repo.scriptinfo.ScriptInfo._get_jupyter_notebook_filename() returns None

right now I can't figure out how to get the session in order to get the notebook path

but the only exception handler is for requests.exceptions.SSLError

it does return kernels, just not sessions

seems like it's using None and that doesn't provide the normal api/sessions endpoint - or, it does, but returns an empty list

so notebook path is empty

As another test I ran Jupyter Lab locally using the same custom Docker container that we're using for Sagemaker Studio, and it works great there, just like the native local Jupyter Lab. So it's seemingly not the image, but maybe something to do with how Studio runs it as a kernel.

the problem is here: None

the server_info is

[{'base_url': '/jupyter/default/',
  'hostname': '0.0.0.0',
  'password': False,
  'pid': 9,
  'port': 8888,
  'root_dir': '/home/sagemaker-user',
  'secure': False,
  'sock': '',
  'token': '',
  'url': '

',
  'version': '1.23.2'}]

local Jupyter Lab: