Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello. I'M Trying Out Self-Hosting With The Latest Version. Upon Spinning Up, Clearml-Elastic Errors-Out With:

Hello. I'm trying out self-hosting with the latest version. Upon spinning up, clearml-elastic errors-out with:

"stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:173) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:160) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.17.18.jar:7.17.18]",
"at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:125) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.17.18.jar:7.17.18]",
"Caused by: org.elasticsearch.ElasticsearchException: failed to bind service",
"at org.elasticsearch.node.Node.<init>(Node.java:1089) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.17.18.jar:7.17.18]",
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:169) ~[elasticsearch-7.17.18.jar:7.17.18]",

The compose yml is unchanged, and checking with lsof , ports 9200 and 9300 are free. Changing the ports doesn't resolve the issue.
Any tips?

  
  
Posted 15 days ago
Votes Newest

Answers 11


also check apiserver and elastic logs and find if there are any errors.

  
  
Posted 14 days ago

Try below options:

sudo chown -R 1000:1000 /opt/clearml

in docker-compose.yaml , elasticsearch -> volumes
Change
- /opt/clearml/data/elastic_7:/usr/share/elasticsearch/data
TO
- /opt/clearml/data/elastic_7:/var/lib/elasticsearch/data

  
  
Posted 15 days ago

copy credential from webUI to ~/clearml.conf (linux)

  
  
Posted 13 days ago

i have same issue. solved by option 2.

  
  
Posted 15 days ago

then you delete the browser’s cookie and cache, log in webui again to check if the credential is still there.

  
  
Posted 14 days ago

Should I create the credential at the localhost server ?

  
  
Posted 14 days ago

i found the created credential will be missing.
image

  
  
Posted 14 days ago

first, you need to make sure all containers are running normally. especially the clearml-elastic , this container may restart every few seconds due to some errors. such situation may cause authorization issue.

  
  
Posted 14 days ago

based on my understanding, the key/secret is only for agent services on server side.

  
  
Posted 13 days ago

Hey, thanks. I missed this in the docker configuration. This seems to resolve the problem, however now clearml-agent-services has authorization issues. In docker logs :

Unauthorized (invalid credentials) (failed to locate provided credentials)

I have set the generated credentials in my clearml conf file and the agent-services config in the compose.
image
image

  
  
Posted 15 days ago

hope it’s helpful

  
  
Posted 15 days ago
65 Views
11 Answers
15 days ago
13 days ago
Tags