Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
What Sort Of Integration Is Possible With Clearml And Sagemaker? On The Page

What sort of integration is possible with ClearML and SageMaker? On the page describing ClearML Remote it says:

Create a remote development environment (e.g. AWS SageMaker, GCP CoLab, etc.) on any on-prem machine or any cloud.

But the only mention of SageMaker I see in the docs is the release notes for 0.13 saying "Add support for SageMaker".

I have SageMaker Studio up and running with access to my ClearML server and it's successfully able to log plots and scalars from experiments, but in terms of code it just logs the code used to launch the kernel:

"""Entry point for launching an IPython kernel.
This is separate from the ipykernel package so we can avoid doing imports until
after removing the cwd from sys.path.
"""
import sys

if __name__ == '__main__':
    # Remove the CWD from sys.path while we load stuff.
    # This is added back by InteractiveShellApp.init_path()
    if sys.path[0] == '':
        del sys.path[0]
    from ipykernel import kernelapp as app
    app.launch_new_instance()

Is it possible to capture more than that while using SageMaker?

  
  
Posted 2 years ago
Votes Newest

Answers 77


We will add this to the SDK soon

  
  
Posted 2 years ago

but r.json() is an empty list

  
  
Posted 2 years ago

Try to add here:
None

server_info['url'] = f"http://{server_info['hostname']}:{server_info['port']}/"
  
  
Posted 2 years ago

sounds good!

  
  
Posted 2 years ago

So it's seemingly not the image, but maybe something to do with how Studio runs it as a kernel.

Yeah I think that for some reason it fails detecting this is actually jupyter noteboko (not really sure why), Thank you for double checking on the container !!

  
  
Posted 2 years ago

print(requests.get(url='

print(requests.get(url='
  
  
Posted 2 years ago

Hmm and you are getting empty list for thi one:

server_info['url'] = f"http://{server_info['hostname']}:{server_info['port']}/"
  
  
Posted 2 years ago

At the top there should be the URL of the notebook (I think)

  
  
Posted 2 years ago

sh-4.2$ cat /var/log/studio/kernel_gateway.log | head -n10
{"__timestamp__": "2023-02-23T21:48:28.036559Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0012829303741455078, "method": "GET", "uri": "/api", "status": 200}
{"__timestamp__": "2023-02-23T21:48:39.111068Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0012879371643066406, "method": "GET", "uri": "/api/kernels", "status": 200}
{"__timestamp__": "2023-02-23T21:48:39.116324Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0007715225219726562, "method": "GET", "uri": "/api/terminals", "status": 200}
{"__timestamp__": "2023-02-23T21:48:39.272822Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0007491111755371094, "method": "GET", "uri": "/api/terminals", "status": 200}
{"__timestamp__": "2023-02-23T21:48:43.000795Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 2.539133071899414, "method": "POST", "uri": "/api/kernels", "status": 201}
{"__timestamp__": "2023-02-23T21:48:43.073568Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0013430118560791016, "method": "GET", "uri": "/api/kernels/6ba227af-ff2c-4b20-89ac-86dcac95e2b2", "status": 200}
{"__timestamp__": "2023-02-23T21:48:43.469751Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0013761520385742188, "method": "GET", "uri": "/api/kernels/6ba227af-ff2c-4b20-89ac-86dcac95e2b2", "status": 200}
{"__timestamp__": "2023-02-23T21:48:43.702549Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0013780593872070312, "method": "GET", "uri": "/api/kernels/6ba227af-ff2c-4b20-89ac-86dcac95e2b2", "status": 200}
{"__timestamp__": "2023-02-23T21:48:43.986808Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.0007445812225341797, "method": "GET", "uri": "/api/kernels/6ba227af-ff2c-4b20-89ac-86dcac95e2b2", "status": 200}
{"__timestamp__": "2023-02-23T21:48:43.992860Z", "__schema__": "sagemaker.kg.request.schema", "__schema_version__": 1, "__metadata_version__": 1, "account_id": "", "duration": 0.001028299331665039, "method": "GET", "uri": "/api/kernels", "status": 200}
  
  
Posted 2 years ago

if there are any tests/debugging you'd like me to try, just let me know

  
  
Posted 2 years ago

it does return kernels, just not sessions

  
  
Posted 2 years ago

yeah, even then it'll run but return 0 notebooks

  
  
Posted 2 years ago

image

  
  
Posted 2 years ago

if I use the same kernel there'll be two

  
  
Posted 2 years ago

right now I can't figure out how to get the session in order to get the notebook path

  
  
Posted 2 years ago

What happens when you call:

from clearml.backend_interface.task.repo import ScriptInfo

print(ScriptInfo._ScriptInfo__legacy_jupyter_notebook_server_json_parsing(None))
  
  
Posted 2 years ago

but one possible workaround is to try to figure out if it's running in a gateway and then find the only notebook running on that server

  
  
Posted 2 years ago

image

  
  
Posted 2 years ago

so notebooks ends up empty

  
  
Posted 2 years ago

as best I can tell it'll only have one .ipynb in $HOME with this setup, which may work...

  
  
Posted 2 years ago

Just ran the same notebook in a local Jupyter Lab session and it worked as I expected it might, saving a copy to Artifacts

  
  
Posted 2 years ago

weird that it won't return that single session

  
  
Posted 2 years ago

As in, which tab when I'm viewing the Experiment should I see it on? Should it be code, an artifact, or something else?

  
  
Posted 2 years ago

the server_info is

[{'base_url': '/jupyter/default/',
  'hostname': '0.0.0.0',
  'password': False,
  'pid': 9,
  'port': 8888,
  'root_dir': '/home/sagemaker-user',
  'secure': False,
  'sock': '',
  'token': '',
  'url': '
',
  'version': '1.23.2'}]
  
  
Posted 2 years ago

Yes, I'm running a notebook in Studio. Where should it be captured?

  
  
Posted 2 years ago

the problem is here: None

  
  
Posted 2 years ago

nice! Just tested it on my end as well, looks like it works!

  
  
Posted 2 years ago

This is strange, let me see if we can get around it, because I'm sure it worked 🙂

  
  
Posted 2 years ago

local Jupyter Lab:
image
image
image

  
  
Posted 2 years ago

and this

server_info['url'] = f"http://{server_info['hostname']}:{server_info['port']}/{server_info['base_url']}/"
  
  
Posted 2 years ago
158K Views
77 Answers
2 years ago
2 years ago
Tags
Similar posts