Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello. I'M Trying Out Self-Hosting With The Latest Version. Upon Spinning Up, Clearml-Elastic Errors-Out With:

Hello. I'm trying out self-hosting with the latest version. Upon spinning up, clearml-elastic errors-out with:

"stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:173) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:160) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:125) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.17.18.jar:7.17.18]",
"Caused by: org.elasticsearch.ElasticsearchException: failed to bind service",
"at org.elasticsearch.node.Node.<init>(Node.java:1089) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:169) ~[elasticsearch-7.17.18.jar:7.17.18]",

The compose yml is unchanged, and checking with lsof , ports 9200 and 9300 are free. Changing the ports doesn't resolve the issue.
Any tips?

  
  
Posted one year ago
Votes Newest

Answers 11


copy credential from webUI to ~/clearml.conf (linux)

  
  
Posted one year ago

then you delete the browser’s cookie and cache, log in webui again to check if the credential is still there.

  
  
Posted one year ago

i found the created credential will be missing.
image

  
  
Posted one year ago

Hey, thanks. I missed this in the docker configuration. This seems to resolve the problem, however now clearml-agent-services has authorization issues. In docker logs :

Unauthorized (invalid credentials) (failed to locate provided credentials)

I have set the generated credentials in my clearml conf file and the agent-services config in the compose.
image
image

  
  
Posted one year ago

based on my understanding, the key/secret is only for agent services on server side.

  
  
Posted one year ago

also check apiserver and elastic logs and find if there are any errors.

  
  
Posted one year ago

first, you need to make sure all containers are running normally. especially the clearml-elastic , this container may restart every few seconds due to some errors. such situation may cause authorization issue.

  
  
Posted one year ago

Try below options:

sudo chown -R 1000:1000 /opt/clearml

in docker-compose.yaml , elasticsearch -> volumes
Change
- /opt/clearml/data/elastic_7:/usr/share/elasticsearch/data
TO
- /opt/clearml/data/elastic_7:/var/lib/elasticsearch/data

  
  
Posted one year ago

i have same issue. solved by option 2.

  
  
Posted one year ago

hope it’s helpful

  
  
Posted one year ago

Should I create the credential at the localhost server ?

  
  
Posted one year ago
1K Views
11 Answers
one year ago
one year ago
Tags