hi @<1734020208089108480:profile|WickedHare16> what was the issue here with the api server? im getting the same warning log:
[2024-08-18 14:03:26,881] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
not sure why
@<1523701070390366208:profile|CostlyOstrich36> Found the issues and fixed it
I have also added the certificated to keychain
api server contain these logs
[2024-08-13 18:06:32,523] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:35,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:35,179] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:45,179] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:45,181] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:46,714] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:47,089] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:47,532] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:55,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:55,180] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:07:01,715] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:02,101] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:02,539] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:05,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:07:05,180] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
It seems that the SDK can't reach the API server. Are you seeing anything in the API server logs? Is it possible you're being blocked by an internal firewall?
WebApp: 1.16.0-494 • Server: 1.16.0-494 • API: 2.30
Hi @<1523701070390366208:profile|CostlyOstrich36>
Getting issue in setting python local config using clearml-init
Verifying credentials ...
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)'))': /auth.login
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)'))': /auth.login
Error: could not verify credentials: key=<key> secret=<secret>
@<1734020208089108480:profile|WickedHare16> , many different things, RBAC, users & groups, K8s dedicated support with advanced features, HyperDatasets, SSO/LDAP integration, dedicated support, dynamic GPU allocation, advanced GPU fractioning on top of K8s and much more.
You can see a more detailed list here - None
I would suggest contacting sales@clear.ml for more information 🙂
And what other features are not available in open source?
In the open source you don't have users & groups, user management is done via fixed users - None
What errors are you seeing in the apiserver pod?
I have setup the infra in k8s and it working now. Now i am trying to create the new user, but the User & group tab in settings page is not visible to any pre-existing users
@<1523701070390366208:profile|CostlyOstrich36> thanks for doc
@<1734020208089108480:profile|WickedHare16> , I suggest going through the documentation and seeing how each part works with the other
None
@<1523701070390366208:profile|CostlyOstrich36> can you also explain the dependency of these components with each other
mongodb is not getting failed. with log
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m[1mWelcome to the Bitnami mongodb container[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0mSubscribe to project updates by watching [1m
[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0mSubmit issues and feature requests at [1m
[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m[38;5;2mINFO [0m ==> ** Starting MongoDB setup **
[38;5;6mmongodb [38;5;5m12:59:37.87 [0m[38;5;2mINFO [0m ==> Validating settings in MONGODB_* env vars...
[38;5;6mmongodb [38;5;5m12:59:37.89 [0m[38;5;3mWARN [0m ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
[38;5;6mmongodb [38;5;5m12:59:37.90 [0m[38;5;2mINFO [0m ==> Initializing MongoDB...
[38;5;6mmongodb [38;5;5m12:59:37.93 [0m[38;5;2mINFO [0m ==> Deploying MongoDB with persisted data...
[38;5;6mmongodb [38;5;5m12:59:37.94 [0m[38;5;2mINFO [0m ==> ** MongoDB setup finished! **
[38;5;6mmongodb [38;5;5m12:59:37.94 [0m[38;5;2mINFO [0m ==> ** Starting MongoDB **
{"t":{"$date":"2024-08-12T12:59:37.964+00:00"},"s":"I", "c":"CONTROL", "id":20698, "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2024-08-12T12:59:37.965+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}}
{"t":{"$date":"2024-08-12T12:59:37.965+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2024-08-12T12:59:37.966+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2024-08-12T12:59:37.966+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","ns":"config.tenantMigrationDonors"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"REPL", "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","ns":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"CONTROL", "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/bitnami/mongodb/data/db","architecture":"64-bit","host":"clearml-mongodb-5f776cb685-52ctl"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"5.0.10","gitVersion":"bbf5bc7e16d1713c9349a09adf4901ca37472e66","openSSLVersion":"OpenSSL 1.1.1n 15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"debian11","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"PRETTY_NAME=\"Debian GNU/Linux 11 (bullseye)\"","version":"Kernel 5.10.219-208.866.amzn2.x86_64"}}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/opt/bitnami/mongodb/conf/mongodb.conf","net":{"bindIp":"*","ipv6":false,"port":27017,"unixDomainSocket":{"enabled":true,"pathPrefix":"/opt/bitnami/mongodb/tmp"}},"processManagement":{"fork":false,"pidFilePath":"/opt/bitnami/mongodb/tmp/mongodb.pid"},"security":{"authorization":"disabled"},"setParameter":{"enableLocalhostAuthBypass":"true"},"storage":{"dbPath":"/bitnami/mongodb/data/db","directoryPerDB":false,"journal":{"enabled":true}},"systemLog":{"destination":"file","logAppend":true,"logRotate":"reopen","path":"/opt/bitnami/mongodb/logs/mongodb.log","quiet":false,"verbosity":0}}}}
{"t":{"$date":"2024-08-12T12:59:37.975+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=7339M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2024-08-12T12:59:38.554+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:554085][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.563+00:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.297"}}
{"t":{"$date":"2024-08-12T12:59:38.570+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:570973][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.606+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:606551][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.615+00:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.298"}}
{"t":{"$date":"2024-08-12T12:59:38.620+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:620755][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.657+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:657849][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.667+00:00"},"s":"I", "c":"STORAGE", "id":22430, "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.299"}}
{"t":{"$date":"2024-08-12T12:59:38.673+00:00"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:673261][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F", "c":"STORAGE", "id":28595, "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"1: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F", "c":"-", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":687}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F", "c":"-", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
Hi @<1734020208089108480:profile|WickedHare16> , what issues are you facing?