Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi Everyone!. I Was Trying To Setup Clearml In Kubernetes Using Helm Chart

👋 Hi everyone!.
I was trying to setup clearml in kubernetes using helm chart 7.11.1 . But getting issues with installation with default values of config None . Can you help me, in setting correct values.

  
  
Posted 3 months ago
Votes Newest

Answers 17


Hi @<1734020208089108480:profile|WickedHare16> , what issues are you facing?

  
  
Posted 3 months ago

mongodb is not getting failed. with log

[38;5;6mmongodb [38;5;5m12:59:37.86 [0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m[1mWelcome to the Bitnami mongodb container[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0mSubscribe to project updates by watching [1m
[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0mSubmit issues and feature requests at [1m
[0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m
[38;5;6mmongodb [38;5;5m12:59:37.86 [0m[38;5;2mINFO [0m ==> ** Starting MongoDB setup **
[38;5;6mmongodb [38;5;5m12:59:37.87 [0m[38;5;2mINFO [0m ==> Validating settings in MONGODB_* env vars...
[38;5;6mmongodb [38;5;5m12:59:37.89 [0m[38;5;3mWARN [0m ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes. For safety reasons, do not use this flag in a production environment.
[38;5;6mmongodb [38;5;5m12:59:37.90 [0m[38;5;2mINFO [0m ==> Initializing MongoDB...
[38;5;6mmongodb [38;5;5m12:59:37.93 [0m[38;5;2mINFO [0m ==> Deploying MongoDB with persisted data...
[38;5;6mmongodb [38;5;5m12:59:37.94 [0m[38;5;2mINFO [0m ==> ** MongoDB setup finished! **

[38;5;6mmongodb [38;5;5m12:59:37.94 [0m[38;5;2mINFO [0m ==> ** Starting MongoDB **

{"t":{"$date":"2024-08-12T12:59:37.964+00:00"},"s":"I",  "c":"CONTROL",  "id":20698,   "ctx":"-","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2024-08-12T12:59:37.965+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":13},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":13},"outgoing":{"minWireVersion":0,"maxWireVersion":13},"isInternalClient":true}}}
{"t":{"$date":"2024-08-12T12:59:37.965+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2024-08-12T12:59:37.966+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2024-08-12T12:59:37.966+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","ns":"config.tenantMigrationDonors"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","ns":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/bitnami/mongodb/data/db","architecture":"64-bit","host":"clearml-mongodb-5f776cb685-52ctl"}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"5.0.10","gitVersion":"bbf5bc7e16d1713c9349a09adf4901ca37472e66","openSSLVersion":"OpenSSL 1.1.1n  15 Mar 2022","modules":[],"allocator":"tcmalloc","environment":{"distmod":"debian11","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"PRETTY_NAME=\"Debian GNU/Linux 11 (bullseye)\"","version":"Kernel 5.10.219-208.866.amzn2.x86_64"}}}
{"t":{"$date":"2024-08-12T12:59:37.967+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/opt/bitnami/mongodb/conf/mongodb.conf","net":{"bindIp":"*","ipv6":false,"port":27017,"unixDomainSocket":{"enabled":true,"pathPrefix":"/opt/bitnami/mongodb/tmp"}},"processManagement":{"fork":false,"pidFilePath":"/opt/bitnami/mongodb/tmp/mongodb.pid"},"security":{"authorization":"disabled"},"setParameter":{"enableLocalhostAuthBypass":"true"},"storage":{"dbPath":"/bitnami/mongodb/data/db","directoryPerDB":false,"journal":{"enabled":true}},"systemLog":{"destination":"file","logAppend":true,"logRotate":"reopen","path":"/opt/bitnami/mongodb/logs/mongodb.log","quiet":false,"verbosity":0}}}}
{"t":{"$date":"2024-08-12T12:59:37.975+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=7339M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],"}}
{"t":{"$date":"2024-08-12T12:59:38.554+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:554085][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.563+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.297"}}
{"t":{"$date":"2024-08-12T12:59:38.570+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:570973][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.606+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:606551][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.615+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.298"}}
{"t":{"$date":"2024-08-12T12:59:38.620+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:620755][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.657+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":17,"message":"[1723467578:657849][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: File exists"}}
{"t":{"$date":"2024-08-12T12:59:38.667+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"unexpected file WiredTiger.wt found, renamed to WiredTiger.wt.299"}}
{"t":{"$date":"2024-08-12T12:59:38.673+00:00"},"s":"E",  "c":"STORAGE",  "id":22435,   "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":1,"message":"[1723467578:673261][1:0x7f0ba1978c80], connection: __posix_open_file, 808: /bitnami/mongodb/data/db/WiredTiger.wt: handle-open: open: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"W",  "c":"STORAGE",  "id":22347,   "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F",  "c":"STORAGE",  "id":28595,   "ctx":"initandlisten","msg":"Terminating.","attr":{"reason":"1: Operation not permitted"}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F",  "c":"-",        "id":23091,   "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":28595,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":687}}
{"t":{"$date":"2024-08-12T12:59:38.678+00:00"},"s":"F",  "c":"-",        "id":23092,   "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}

image

  
  
Posted 3 months ago

@<1523701070390366208:profile|CostlyOstrich36> can you also explain the dependency of these components with each other

  
  
Posted 3 months ago

@<1734020208089108480:profile|WickedHare16> , I suggest going through the documentation and seeing how each part works with the other
None

  
  
Posted 3 months ago

@<1523701070390366208:profile|CostlyOstrich36> thanks for doc

  
  
Posted 3 months ago

I have setup the infra in k8s and it working now. Now i am trying to create the new user, but the User & group tab in settings page is not visible to any pre-existing users

  
  
Posted 3 months ago

getting this on creating new creadentials
image

  
  
Posted 3 months ago

In the open source you don't have users & groups, user management is done via fixed users - None

What errors are you seeing in the apiserver pod?

  
  
Posted 3 months ago

And what other features are not available in open source?

  
  
Posted 3 months ago

@<1734020208089108480:profile|WickedHare16> , many different things, RBAC, users & groups, K8s dedicated support with advanced features, HyperDatasets, SSO/LDAP integration, dedicated support, dynamic GPU allocation, advanced GPU fractioning on top of K8s and much more.

You can see a more detailed list here - None

I would suggest contacting sales@clear.ml for more information 🙂

  
  
Posted 3 months ago

Hi @<1523701070390366208:profile|CostlyOstrich36>
Getting issue in setting python local config using clearml-init

Verifying credentials ...
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)'))': /auth.login
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate (_ssl.c:1007)'))': /auth.login
Error: could not verify credentials: key=<key> secret=<secret>
  
  
Posted 3 months ago

WebApp: 1.16.0-494 • Server: 1.16.0-494 • API: 2.30

  
  
Posted 3 months ago

It seems that the SDK can't reach the API server. Are you seeing anything in the API server logs? Is it possible you're being blocked by an internal firewall?

  
  
Posted 3 months ago

api server contain these logs

[2024-08-13 18:06:32,523] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:35,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:35,179] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:45,179] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:45,181] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:46,714] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:47,089] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:47,532] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:06:55,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:06:55,180] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:07:01,715] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:02,101] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:02,539] [9] [WARNING] [clearml.service_repo] Returned 400 for in 0ms, msg=Invalid request path /
[2024-08-13 18:07:05,178] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
[2024-08-13 18:07:05,180] [9] [INFO] [clearml.service_repo] Returned 200 for debug.ping in 0ms
  
  
Posted 3 months ago

I have also added the certificated to keychain

  
  
Posted 3 months ago

@<1523701070390366208:profile|CostlyOstrich36> Found the issues and fixed it

  
  
Posted 3 months ago

hi @<1734020208089108480:profile|WickedHare16> what was the issue here with the api server? im getting the same warning log:

[2024-08-18 14:03:26,881] [9] [WARNING] [clearml.service_repo] Returned 400 for  in 0ms, msg=Invalid request path /

not sure why

  
  
Posted 3 months ago
255 Views
17 Answers
3 months ago
3 months ago
Tags
Similar posts