here's my script:

#!/bin/bash
echo "******************** Starting Agent ********************"
echo "******************** Getting ENV Variables ********************"
source /etc/profile.d/env-vars.sh

# test that we can access the API
echo "******************** Waiting for ${CLEARML_API_HOST} connectivity ********************"
curl --retry 10 --retry-delay 10 --retry-connrefused ${CLEARML_API_HOST}/debug.ping

# start the agent
for i in $(seq 1 ${NUM_WORKERS})
do 
    export CLEARML_WORK...

so notebooks ends up empty

it does return kernels, just not sessions

the problem is here: None

I can get it to run up to here: None

weird that it won't return that single session

that fails

lots of things like {"__timestamp__": "2023-02-23T23:49:23.285946Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0007679462432861328, "method": "GET", "uri": "/api/kernels/6ba227af-ff2c-4b20-89ac-86dcac95e2b2", "status": 200}

and cat /var/log/studio/kernel_gateway.log | grep ipynb comes up empty

but even then the sessions endpoint is still empty

and that requests.get() throws an exception:

ConnectionError: HTTPConnectionPool(host='default', port=8888): Max retries exceeded with url: /jupyter/default/api/sessions (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f7ba9cadc30>: Failed to establish a new connection: [Errno -2] Name or service not known'))

Just ran the same notebook in a local Jupyter Lab session and it worked as I expected it might, saving a copy to Artifacts

poking around a little bit, and clearml.backend_interface.task.repo.scriptinfo.ScriptInfo._get_jupyter_notebook_filename() returns None

at least in 2018 it returned sessions! None

one possibility for getting the notebook filepath is finding and parsing /home/sagemaker-user/.jupyter/lab/workspaces/default-37a8.jupyterlab-workspace I think, but I don't know if I can tie that to a specific session

but the call to jupyter_server.serverapp.list_running_servers() does return the server

local Jupyter Lab:

thanks for the thoughtful response, @<1523701205467926528:profile|AgitatedDove14> ! I think I'll need to test out some workflows to see what works

I could just loop through and create separate pipelines with different parameters, but seems sort of inefficient. the hyperparameter optimization might actually work in this case utilizing grid search, but seems like kind of a hack

cool, thanks! the first one was what I had thought of but seemed unpythonic, so I'll give the second a shot

those look like linear DAGs to me, but maybe I'm missing something. I'm thinking something like the map operator in Prefect where I can provide an array of ["A", "B", "C"] and run the steps outlined with dotted lines independently for each of those are arguments

but r.json() is an empty list

the server_info is

[{'base_url': '/jupyter/default/',
  'hostname': '0.0.0.0',
  'password': False,
  'pid': 9,
  'port': 8888,
  'root_dir': '/home/sagemaker-user',
  'secure': False,
  'sock': '',
  'token': '',
  'url': '

',
  'version': '1.23.2'}]

I'll give it a shot and see! Just setting up a test server now, so it's still a hypothetical question just from reading the docs so far

So I'm thinking maybe a Project for each thing we're forecasting, and then new Tasks for each time we run it

My use case is running forecasting models in production across multiple businesses

I'm not sure if Subprojects will work for that - can you use the Web UI to compare the artifacts from two separate subprojects?

awesome, I'll test it out - thanks for the tips!

But we're also testing out new models all the time, which are typically implemented as git branches - they run on the same set of inputs but don't output their results into production

And then we want to compare backtests or just this week's estimates across multiple of those models/branches