Examples: query, "exact match", wildcard*, wild?ard, wild*rd
Fuzzy search: cake~ (finds cakes, bake)
Term boost: "red velvet"^4, chocolate^2
Field grouping: tags:(+work -"fun-stuff")
Escaping: Escape characters +-&|!(){}[]^"~*?:\ with \, e.g. \+
Range search: properties.timestamp:[1587729413488 TO *] (inclusive), properties.title:{A TO Z}(excluding A and Z)
Combinations: chocolate AND vanilla, chocolate OR vanilla, (chocolate OR vanilla) NOT "vanilla pudding"
Field search: properties.title:"The Title" AND text
Answered
Hello! We Are Trying To Upgrade From Trains Server 15.1 To 16.1 Using Docker, But Are Running Into A Permission Error:

Hello! We are trying to upgrade from Trains server 15.1 to 16.1 using Docker, but are running into a permission error:
trains-elastic | "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];"(see follow-up comment for full Traceback)

We followed the migration guide, and it seemed like things went well.

In the docker-compose.yml you have the following lines:
volumes: - /opt/trains/data/elastic_7:/usr/share/elasticsearch/dataSo we thought the permission has to do with this elastic_7 folder. However, even if we rename the existing one, and therefore Docker creates this elastic_7 folder (as root user), the same permission error shows up. elastic_7 is also empty.

Docker should run as root on our setup, so that shouldn't be the issue.
Any idea what the problem could be?

(We're on Ubuntu 18.04)

  
  
Posted 3 years ago
Votes Newest

Answers 10


/opt/trains/ :

$ ls -al total 120 drwxrwsrwx 7 root miniconda 4096 Nov 2 18:15 . drwxr-xr-x 15 root root 4096 Oct 5 15:12 .. drwxrwxrwx 38 root miniconda 4096 Nov 2 18:15 agent drwxrwxrwx 2 root miniconda 4096 Jun 19 14:43 config drwxrwxrwx 8 root miniconda 4096 Nov 2 18:11 data -rwxrwxrwx 1 root miniconda 4383 Jun 19 14:46 docker-compose_0.15.0.yml -rwxrwxrwx 1 root miniconda 4375 Jun 26 15:06 docker-compose_0.15.1.yml -rwxrwxrwx 1 root miniconda 4324 Nov 2 18:15 docker-compose.yml -rwxrwxrwx 1 root miniconda 13248 Oct 30 18:12 elastic_upgrade.py drwxrwxrwx 2 root miniconda 4096 Oct 30 18:40 logs -rwxrwxrwx 1 root miniconda 8851 Oct 30 17:55 trains-server-0.16.0-migration.zip drwxrwxrwx 3 root miniconda 4096 Oct 30 18:13 upgrade_files -rwxrwxrwx 1 root miniconda 303 Oct 30 18:02 upgrade_to_7_2020-10-30_18-02-01.log -rwxrwxrwx 1 root miniconda 221 Oct 30 18:03 upgrade_to_7_2020-10-30_18-03-42.log -rwxrwxrwx 1 root miniconda 221 Oct 30 18:06 upgrade_to_7_2020-10-30_18-06-34.log -rwxrwxrwx 1 root miniconda 22707 Oct 30 18:16 upgrade_to_7_2020-10-30_18-12-07.log
/opt/trains/data/elastic_7 :
$ ls -al total 12 drwxrwxrwx 3 root miniconda 4096 Oct 30 18:12 . drwxrwxrwx 8 root miniconda 4096 Nov 2 18:11 .. drwxrwxrwx 3 root miniconda 4096 Oct 30 18:12 nodesCurrently I've set it back to 777

  
  
Posted 3 years ago

trains-elastic exited with code 1 trains-elastic | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. trains-elastic | {"type": "server", "timestamp": "2020-11-02T08:04:57,699Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "trains", "node.name": "trains", "message": "uncaught exception in thread [main]", trains-elastic | "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "Caused by: org.elasticsearch.ElasticsearchException: failed to bind service", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:615) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "... 6 more", trains-elastic | "Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes", trains-elastic | "at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389) ~[?:?]", trains-elastic | "at java.nio.file.Files.createDirectory(Files.java:693) ~[?:?]", trains-elastic | "at java.nio.file.Files.createAndCheckIsDirectory(Files.java:800) ~[?:?]", trains-elastic | "at java.nio.file.Files.createDirectories(Files.java:786) ~[?:?]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:211) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:271) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:277) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "... 6 more"] } trains-elastic | uncaught exception in thread [main] trains-elastic | ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes]; trains-elastic | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes trains-elastic | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) trains-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) trains-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) trains-elastic | at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389) trains-elastic | at java.base/java.nio.file.Files.createDirectory(Files.java:693) trains-elastic | at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:800) trains-elastic | at java.base/java.nio.file.Files.createDirectories(Files.java:786) trains-elastic | at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274) trains-elastic | at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:211) trains-elastic | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:271) trains-elastic | at org.elasticsearch.node.Node.<init>(Node.java:277) trains-elastic | at org.elasticsearch.node.Node.<init>(Node.java:257) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) trains-elastic | at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) trains-elastic | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) trains-elastic | at org.elasticsearch.cli.Command.main(Command.java:90) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) trains-elastic | For complete error details, refer to the log at /usr/share/elasticsearch/logs/trains.log

  
  
Posted 3 years ago

AppetizingMouse58 If I:
sudo chmod 771 -R /opt/trains/(taking all permission away from other except execution)
The file permission error comes back, even though everything is under the root user.

  
  
Posted 3 years ago

Ok it's that the user group also has to be root. I ran the following:
sudo chmod 775 -R /opt/trains/ sudo chown -R root:root /opt/trainsand it works.

It seems that it has to be 775 with both user and group as root. E.g. 771 does not work, because than the docker command has to be used with sudo (if I want to use my default sudo-user account)

  
  
Posted 3 years ago

Can you run 'ls -al' in the /opt/trains/data folder and also in the /opt/trains/data/elastic_7 folder and send the output?

  
  
Posted 3 years ago

Ok, it was indeed something with permission. When I chown everything to root (1000) and chmod 777 it worked. 777 is of course not desirable, so I'm going to narrow it down now.

Thank you for the reply! The migration indeed created this elastic_7 folder.

  
  
Posted 3 years ago

If you run the following command 'sudo chown -R 1000:1000 /opt/trains' does it change anything?

  
  
Posted 3 years ago

Is it possible it's not just about the root user, but also the root group?

  
  
Posted 3 years ago

Same problem with 775

  
  
Posted 3 years ago

Hi DefeatedCrab47 , ES docker requires that it is data folder belongs to 1000:1000 user and group. If you want to transfer your existing data from trains 15.1 then please follow the guide https://allegro.ai/docs/deploying_trains/trains_server_es7_migration/
The script that is run in this guide should create elastic_7 folder with the correct permissions and transfer all your existing data

  
  
Posted 3 years ago
662 Views
10 Answers
3 years ago
one year ago
Tags