Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hi, We'Ve Been Trying To Run The Same Experiments As We Did At Previous Times But Have Been Getting Configuration Errors After Credentials Errors Over And Over. I'Ve Went Ahead And Replaced All Of Our Existing Credentials With New Ones, Went Through The

Hi,
we've been trying to run the same experiments as we did at previous times
but have been getting configuration errors after credentials errors over and over.

I've went ahead and replaced all of our existing credentials with new ones,
went through the clearml-init process and reconfiguration, on multiple machines, none of which worked.

Any help would be appreciated.

  
  
Posted 10 months ago
Votes Newest

Answers 33


but I've tried passing secret key and token directly as env variable as per the doc

  
  
Posted 10 months ago

thank you for your support

  
  
Posted 10 months ago

Yep 🙂

  
  
Posted 10 months ago

I suspect you might be using some wrong configuration file or some wrong/old credentials. Can you try on a brand new environment please? I would maybe even suggest testing on a different machine/user.

I'd suggest using CLEARML_CONFIG_FILE to point directly to the config file. You can also use other env variables to "circumvent" the config file in order to debug - None

  
  
Posted 10 months ago

an anomaly that occured passed the point where the workspace was transferred is that comapnies workspace overriden my personal user's workspace

  
  
Posted 10 months ago

Like mentioned I've tried doing that,
O

  
  
Posted 10 months ago

Happy to help 🙂

  
  
Posted 10 months ago

which resulted in the error as seen in configerror2osenv.txt

  
  
Posted 10 months ago

I'm assuminmg 10:57 is your guys attempt

  
  
Posted 10 months ago

not sure, currently only tried through my user

  
  
Posted 10 months ago

Hi @<1653932438834909184:profile|LackadaisicalSwallow14> , can you add some information please? Are you using a self deployed server or the community? Has something changed recently? Is the issue occurring for all users or only specific ones?

  
  
Posted 10 months ago

Does the issue occur for all users in the workspace?

  
  
Posted 10 months ago

this doesn't explain as to why the env variables didn't work though

Maybe you defined the env variables outside the container? Maybe incorrect usage on your end? The env variables work when properly configured.

My guess would be something related to your environments.

  
  
Posted 10 months ago

but 19:15 is mine from yesterday

  
  
Posted 10 months ago

I've tried the steps mentioned on two different machines, I don't mind rebooting the second machine and retrying the steps including the CLEARML_CONFIG_FILE env variable

  
  
Posted 10 months ago

Have you played with the env variables?

  
  
Posted 10 months ago

Ok so that's normal

  
  
Posted 10 months ago

This the internal test made by the guys. Everything worked from a fresh properly configured station.

  
  
Posted 10 months ago

the 401 error was with the second set of credentials, when used as OS variables,
before that I've also tried as config file but got the same error as on the first machine (configerror1)

  
  
Posted 10 months ago

I have not tried though to use the CLEARML_CONFIG_FILE env varfiable

  
  
Posted 10 months ago

but the probem seems to have beensolved now and it was on our end

  
  
Posted 10 months ago

this doesn't explain as to why the env variables didn't work though

  
  
Posted 10 months ago

also, at the configuration page on the website, the tokens are updated as being used, when I attempt at running the experiments
image

  
  
Posted 10 months ago

And what was the result from 19:15 yesterday? The 401 error? Please note that's a different set of credentials

  
  
Posted 10 months ago

Hi, ofcourse:
we're using the clearml website,
two things have changed recently - we've had a change in team lead, and after we contacted some one on your side, they've moved ownership of our companies workspace to another persons user. second thing changed is we've registered to Pro services.

these are some error logs and the config process stdout

  
  
Posted 10 months ago

meaning instead of having a personal workspace and a company worksapce as was before the transfer, my user currently only has single workspace

  
  
Posted 10 months ago

probably

  
  
Posted 10 months ago

another thing that seems weird: I looked at the configuration page from another user in our workspace, on their page when they look at the companies workspace credentials, you can only see a single (different) credential token created, are these pages per user? or if they're per workspace I would expect to see the same tokens on both users view of the workspace configurations

  
  
Posted 10 months ago

Ok, I've found the problem:
I would run the experiment in a docker container, the credentials from the machine were mounted correctly to the container at first to /root/clearml.conf, later in the experiment the dockerfile changed such to create a user for experiment to run in (to prevent later root perms problems.) which then probably caused the problem due to the user not getting the creds from /root/clearml.conf

  
  
Posted 10 months ago

They're per user per workspace. So each user will have their own unique credentials for every different workspace

  
  
Posted 10 months ago
32K Views
33 Answers
10 months ago
10 months ago
Tags