trains-elastic exited with code 1 trains-elastic | OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. trains-elastic | {"type": "server", "timestamp": "2020-11-02T08:04:57,699Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "trains", "node.name": "trains", "message": "uncaught exception in thread [main]", trains-elastic | "stacktrace": ["org.elasticsearch.bootstrap.StartupException: ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) ~[elasticsearch-cli-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "Caused by: org.elasticsearch.ElasticsearchException: failed to bind service", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:615) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "... 6 more", trains-elastic | "Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes", trains-elastic | "at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]", trains-elastic | "at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389) ~[?:?]", trains-elastic | "at java.nio.file.Files.createDirectory(Files.java:693) ~[?:?]", trains-elastic | "at java.nio.file.Files.createAndCheckIsDirectory(Files.java:800) ~[?:?]", trains-elastic | "at java.nio.file.Files.createDirectories(Files.java:786) ~[?:?]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:211) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:271) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:277) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.node.Node.<init>(Node.java:257) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.6.2.jar:7.6.2]", trains-elastic | "... 6 more"] } trains-elastic | uncaught exception in thread [main] trains-elastic | ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes]; trains-elastic | Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes trains-elastic | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) trains-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) trains-elastic | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) trains-elastic | at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389) trains-elastic | at java.base/java.nio.file.Files.createDirectory(Files.java:693) trains-elastic | at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:800) trains-elastic | at java.base/java.nio.file.Files.createDirectories(Files.java:786) trains-elastic | at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:274) trains-elastic | at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:211) trains-elastic | at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:271) trains-elastic | at org.elasticsearch.node.Node.<init>(Node.java:277) trains-elastic | at org.elasticsearch.node.Node.<init>(Node.java:257) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) trains-elastic | at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) trains-elastic | at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) trains-elastic | at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) trains-elastic | at org.elasticsearch.cli.Command.main(Command.java:90) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) trains-elastic | at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) trains-elastic | For complete error details, refer to the log at /usr/share/elasticsearch/logs/trains.log
Hi DefeatedCrab47 , ES docker requires that it is data folder belongs to 1000:1000 user and group. If you want to transfer your existing data from trains 15.1 then please follow the guide https://allegro.ai/docs/deploying_trains/trains_server_es7_migration/
The script that is run in this guide should create elastic_7 folder with the correct permissions and transfer all your existing data
Ok, it was indeed something with permission. When I chown everything to root (1000) and chmod 777 it worked. 777 is of course not desirable, so I'm going to narrow it down now.
Thank you for the reply! The migration indeed created this elastic_7 folder.
AppetizingMouse58 If I:sudo chmod 771 -R /opt/trains/(taking all permission away from other except execution)
The file permission error comes back, even though everything is under the root user.
Can you run 'ls -al' in the /opt/trains/data folder and also in the /opt/trains/data/elastic_7 folder and send the output?
/opt/trains/ :
$ ls -al total 120 drwxrwsrwx 7 root miniconda 4096 Nov 2 18:15 . drwxr-xr-x 15 root root 4096 Oct 5 15:12 .. drwxrwxrwx 38 root miniconda 4096 Nov 2 18:15 agent drwxrwxrwx 2 root miniconda 4096 Jun 19 14:43 config drwxrwxrwx 8 root miniconda 4096 Nov 2 18:11 data -rwxrwxrwx 1 root miniconda 4383 Jun 19 14:46 docker-compose_0.15.0.yml -rwxrwxrwx 1 root miniconda 4375 Jun 26 15:06 docker-compose_0.15.1.yml -rwxrwxrwx 1 root miniconda 4324 Nov 2 18:15 docker-compose.yml -rwxrwxrwx 1 root miniconda 13248 Oct 30 18:12 elastic_upgrade.py drwxrwxrwx 2 root miniconda 4096 Oct 30 18:40 logs -rwxrwxrwx 1 root miniconda 8851 Oct 30 17:55 trains-server-0.16.0-migration.zip drwxrwxrwx 3 root miniconda 4096 Oct 30 18:13 upgrade_files -rwxrwxrwx 1 root miniconda 303 Oct 30 18:02 upgrade_to_7_2020-10-30_18-02-01.log -rwxrwxrwx 1 root miniconda 221 Oct 30 18:03 upgrade_to_7_2020-10-30_18-03-42.log -rwxrwxrwx 1 root miniconda 221 Oct 30 18:06 upgrade_to_7_2020-10-30_18-06-34.log -rwxrwxrwx 1 root miniconda 22707 Oct 30 18:16 upgrade_to_7_2020-10-30_18-12-07.log/opt/trains/data/elastic_7 :$ ls -al total 12 drwxrwxrwx 3 root miniconda 4096 Oct 30 18:12 . drwxrwxrwx 8 root miniconda 4096 Nov 2 18:11 .. drwxrwxrwx 3 root miniconda 4096 Oct 30 18:12 nodesCurrently I've set it back to 777
Is it possible it's not just about the root user, but also the root group?
If you run the following command 'sudo chown -R 1000:1000 /opt/trains' does it change anything?
Ok it's that the user group also has to be root. I ran the following:sudo chmod 775 -R /opt/trains/ sudo chown -R root:root /opt/trainsand it works.
It seems that it has to be 775 with both user and group as root. E.g. 771 does not work, because than the docker command has to be used with sudo (if I want to use my default sudo-user account)