As in, which tab when I'm viewing the Experiment should I see it on? Should it be code, an artifact, or something else?

looks like the same as in server_info

one possibility for getting the notebook filepath is finding and parsing /home/sagemaker-user/.jupyter/lab/workspaces/default-37a8.jupyterlab-workspace I think, but I don't know if I can tie that to a specific session

as best I can tell it'll only have one .ipynb in $HOME with this setup, which may work...

I'm not sure if Subprojects will work for that - can you use the Web UI to compare the artifacts from two separate subprojects?

So I'm thinking maybe a Project for each thing we're forecasting, and then new Tasks for each time we run it

cool, thanks! the first one was what I had thought of but seemed unpythonic, so I'll give the second a shot

SageMaker Studio:

done: None

which I looked at previously to see if I could import sagemaker.kg or kernelgateway or something, but no luck

I additionally tried using a Sagemaker Notebook instance, to see if it was the kernel dockerization that Studio uses that was messing things up. But it seems to actually log less information from a Notebook instance vs Studio .


![image](https://clearml-web-assets.s3.amazonaws.com/scoold/im...

But we're also testing out new models all the time, which are typically implemented as git branches - they run on the same set of inputs but don't output their results into production

I'll give it a shot and see! Just setting up a test server now, so it's still a hypothetical question just from reading the docs so far

I could just loop through and create separate pipelines with different parameters, but seems sort of inefficient. the hyperparameter optimization might actually work in this case utilizing grid search, but seems like kind of a hack

and that requests.get() throws an exception:

ConnectionError: HTTPConnectionPool(host='default', port=8888): Max retries exceeded with url: /jupyter/default/api/sessions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ba9cadc30>: Failed to establish a new connection: [Errno -2] Name or service not known'))

None

thanks for the thoughtful response, @<1523701205467926528:profile|AgitatedDove14> ! I think I'll need to test out some workflows to see what works

if I add the base_url it's not found

sounds good, thanks!

right now I'm doing

# clear existing configs in case we're rerunning the notebook
if hydra.core.global_hydra.GlobalHydra().is_initialized():
    hydra.core.global_hydra.GlobalHydra.instance().clear()

# initialize Hydra
hydra.initialize(
    version_base=None,
    config_path=".",
    job_name="test_app",
)

# use the compose API since we're running in a notebook instead of __main__
cfg = hydra.compose(
    config_name="config", 
    overrides=[],
)

# report in ClearML Config UI
clearm...

nice! Just tested it on my end as well, looks like it works!

I've poked around both the internal URL that Jupyter kernel is running on and some of the files in /sagemaker/.jupyter but no luck so far - I can find plenty of kernel info, but not session

weird that it won't return that single session

sh-4.2$ cat /var/log/studio/kernel_gateway.log | head -n10
{"__timestamp__": "2023-02-23T21:48:28.036559Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0012829303741455078, "method": "GET", "uri": "/api", "status": 200}
{"__timestamp__": "2023-02-23T21:48:39.111068Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.00128793...

cool, if I call clearml.binding.hydra_bind.PatchHydra._register_omegaconf(cfg) directly inside the notebook I can manually cause it to log the Hydra config, it just doesn't seem to autodetect if you're doing a manual call to Hydra Compose

yeah, even then it'll run but return 0 notebooks

api/kernels does report back the active kernel, but doesn't give notebook paths or anything

awesome, I'll test it out - thanks for the tips!