Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi There! I Want To Set Up Clearml On A Kubernetes Cluster. The Whole Setup Including Authentication Of Fixed Users Seems To Work Fine. I Can Log In On The Webapp And Generate Credentials For Connecting My Local Clearml Installation. The Clearml-Init Also

Hi there! I want to set up clearml on a kubernetes cluster. The whole setup including authentication of fixed users seems to work fine. I can log in on the webapp and generate credentials for connecting my local clearml installation. The clearml-init also works but when I want to create an initial dataset using clearml-data create --project "test project" --name "test name" I get an error Error: Failed getting token (error 401 from None .*****.com): Unauthorized (invalid credentials) (failed to locate provided credentials) . Does anybody have an idea how to fix this?

  
  
Posted 8 months ago
Votes Newest

Answers 6


The clearml-data call results in these two lines in the ingress logs. Is that sufficient or would you like to have a larger section of the log?

2024/03/26 16:07:10 [warn] 2879#2879: *1151249 upstream sent duplicate header line: "server: clearml", previous value: "Server: Werkzeug/3.0.1 Python/3.9.18", ignored while reading response header from upstream, client: ***.***.***.22, server: api.clearml.****.com, request: "GET /auth.login HTTP/1.1", upstream: "
", host: "api.clearml.****.com"
77.23.243.22 - CN1YAZ1L7J4CY6GO8WQW [26/Mar/2024:16:07:10 +0000] "GET /auth.login HTTP/1.1" 401 347 "-" "python-requests/2.30.0" 449 0.005 [default-clearml-apiserver-8008] [] 10.213.150.109:8008 347 0.005 401 03f88a11d9264e7f078e8370a5e33dd2
  
  
Posted 8 months ago

Hi @<1523701827080556544:profile|JuicyFox94> I figured out what the problem is! For some recent experimentation I set an acces_key and secret_key as environment variables in my os. When I deleted them everything worked fine so the environment variables overwrote the keys given by the clearml.conf. Is that the desired default behaviour?
And just one tip for everbody having similar problems: Switch to using the SDK instead of the CLI for better debugging. This helped me to find the cause of my issue. :)

  
  
Posted 8 months ago

url returning 401 is same as one exposed by ingress (if you are using them ofc)?

  
  
Posted 8 months ago

ok so we should see the call in ingress controller logs, can you pls share them?

  
  
Posted 8 months ago

Yes it is.

  
  
Posted 8 months ago

adding @<1523701087100473344:profile|SuccessfulKoala55> to the conversation because I’m not totally sure the problem relies on ingress, it looks to be a bad token but it shouldn’t since init was good

  
  
Posted 8 months ago
634 Views
6 Answers
8 months ago
8 months ago
Tags