works like a charm! you guys are the best, as always =)
By the way, best practice for Trains when running on the cloud is to limit external access to 8081
to your office/home IP address
First thing to do - check the EC2 instance security group and make sure port 8081
is open
yeah, that's exactly what I'm looking to right now 😃
The usual suspect is always a Firewall 🙂
Well, this suggests you have some network configuration issue that prevents requests from reaching the fileserver from your machine (and other machines, possible).
Is the server running on the same network, or is it hosted remotely (on AWS EC2, for example)?
OK, so first of all let's go over some basic stuff. I assume you're running Trains Server on some remote machine (i.e. not localhost), I also assume you can SSH to that machine... I also assume you're using docker-compose
to run the server. In the server's console, can you try to:
Run sudo docker ps
and see if the trains-fileserver
docker container is up (and not restarting every few seconds) - its uptime should be roughly the same as that of the trains-apiserver
or any of the other containers. Run curl
http://localhost:8081 from the server's machine and see whether you get an error and not a timeout Open bash
inside the trains-fileserver
docker container using sudo docker exec -it trains-fileserver /bin/bash
and try running curl
http://localhost:8081 there, and see whether you get an error and not a timeout
no, we've only been using metric and model logging so far
DilapidatedDucks58 is this something that used to work (I mean with the exact same use of matplotlib) and is not working now? If so, was there any configuration change in Trains server or your network?
I get "The connection has timed out" when I'm trying to reach 8081 port
FYI matplotlib imshow will create a debug image, and on complex plots the plot might get converted to image. (But shown under the plots section). All in all you might not be aware of it, but you are uploading image to your files server
I'm asking since debug images are uploaded to the trains file server, and matplotlib is sometimes uploaded as an image as well. It might be there's a problem reaching the file server... Can you try browsing to http://YOURIP:8081/ ? You should reach the server but get a file not found error or something similar
Hey DilapidatedDucks58 , are you logging debug images as well?