Highly unlikely as those are all local dns records. 🙂
Hi @<1802511466914385920:profile|PerfectSeaurchin36> , on your points:
- what was the body of the API call, you got 400 so it looks like the body was incorrect
- Not sure what the issue is
Not exactly sure I understand what the issue is, can you please elaborate?
I also tried
Task.set_credentials(
api_host="
",
web_host="
",
files_host="
",
key='E5BYXIM0JXX5N9MRZSHEE2ACKXPTY2',
secret='govErSIYdtu-67EBGVPhriMOOB0QCT_OZ_B2073rGjYO14uYP802dMuOk1_oVV4STxY'
)
Same result
No problem. This is all very ephemeral and will die imminently.
# This will apply to all buckets in this host (unless key/value is specifically provided for a given bucket)
host: "my-minio-host:9000"
You need to add the port for minio
Oh. I see what you mean now.
“To force usage of a non-AWS endpoint, port declaration is always needed (e.g. host: "my-minio-host:9000" ), even for standard ports like 433 for HTTPS (e.g. host: "my-minio-host:433" ).”
It’s running behind an ingress so the port is there.
Hi @<1523701070390366208:profile|CostlyOstrich36>
Thanks for looking at this.
- Pasted the body below
{
"aws": {
"key": "emEijEB2wZtj1rgaUN3y",
"secret": "oTxr3w3nlygv85oULOBWkaJi6Zj41OFBLB1e1m0L",
"region": "",
"token": "",
"use_credentials_chain": false,
"buckets": [{
"bucket": "clearml",
"host": "
",
"key": "emEijEB2wZtj1rgaUN3y",
"secret": "oTxr3w3nlygv85oULOBWkaJi6Zj41OFBLB1e1m0L",
"token": "",
"secure": false,
"region": "",
"verify": null,
"use_credentials_chain": false
}]
},
"google": {
"project": null,
"credentials_json": {},
"buckets": []
},
"azure": {
"containers": []
}
}
I noticed I got this response
{
"meta": {
"id": "a2abdf8724014e01bf8ecd3a98887082",
"trx": "a2abdf8724014e01bf8ecd3a98887082",
"endpoint": {
"name": "storage.set_settings",
"requested_version": "2.31",
"actual_version": "1.0"
},
"result_code": 400,
"result_subcode": 12,
"result_msg": "Validation error (error for field 'google': error for field 'credentials_json': ('value is wrong, expected type \"str\"', {})..)",
"error_stack": null,
"error_data": {}
},
"data": {}
}
I’m not setting google credentials so idk why this would be part of the error
I think I found it.
The google credentials were “set” out of the box but had an empty Object {}. I removed the object and saved and the save was succesful.
I was then able to add my minio credentials and save them as well.
Should I make a bug report for this?
- I managed to recreate it.
#In my jupyter notebook
%pip install clearml --upgrade pip
from clearml import Task, Logger
%env CLEARML_WEB_HOST=
%env CLEARML_API_HOST=
%env CLEARML_FILES_HOST=
%env CLEARML_API_ACCESS_KEY=GZBUVZYG1YHU6QSPMM3PI19EZB51ZY"
%env CLEARML_API_SECRET_KEY=MlirsAtDI_V4_OdzGFk5aB8-Z8ivLV3bZfcLzMzZD6oYSt503Lbmg0HHJDFJW9PGMuE
task = Task.init(project_name='Test', task_name='test')
I get a timeout error
etrying (Retry(total=239, connect=239, read=240, redirect=240, status=240)) after connection broken by 'ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7a91cd5add30>, 'Connection to api-clearml.domain.duckdns.org timed out. (connect timeout=3.0)')': /auth.login
I’m wondering if this is because the url is a local dns set in my unify router and jupyter runs inside the cluster and is trying to resolve it locally?
Is there a way to test if the connection is working?
Fair enough 🙂
BTW I noticed that your minio access is misconfigured, check here for the proper syntax - None
Also. Can I skip setting the bucket name in the UI as it’s part of the URL?
minio creds, server creds, someone with ill intentions could have accessed your cluster by now
Just to make sure the domain is available I did
jovyan@hub-54bbb78ff4-bphnj:/srv/jupyterhub$ dig api-clearml.domain.duckdns.org
; <<>> DiG 9.18.28-1~deb12u2-Debian <<>> api-clearml.domain.duckdns.org
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1149
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 3ac65d14c9554b0f (echoed)
;; QUESTION SECTION:
;api-clearml.domain.duckdns.org. IN A
;; ANSWER SECTION:
api-clearml.domain.duckdns.org. 5 IN A 192.168.70.20
;; Query time: 3 msec
;; SERVER: 10.96.0.10#53(10.96.0.10) (UDP)
;; WHEN: Mon Feb 17 20:29:43 UTC 2025
;; MSG SIZE rcvd: 117
I also couldn’t see any example credentials in the documentation but the bucket is not public so should I assume that credentials will be used if set?
Looks like a network issue.
As a side note, I would suggest removing & revoking all credentials you've pasted here 🙂
It’s a homelab k8s cluster that I tear down for laughs and giggles 🙂
Spun up a jupyterlab locally and was able to connect with clearml. This is obviously a problem with networking on my k8s cluster.