SuccessfulKoala55 Thanks 🙏 I will give it a try tomorrow 🙂
As long as you still have the file server running as part of Trains, and the file server can reach the files using /opt/trains/data/fileserver - it should be OK
Does it still work if I will keep trains.conf like this, and mount the S3 also?
Ohh I understood, so can you give me a short explanation on how to change the meta data?
Hi CooperativeFox72 , I just realized I might have misunderstood you - if you mount /opt/trains/data/fileserver to some S3 bucket and keep using the fileserver configuration in your trains.conf , it should absolutely work....
I was under the impression you wanted to stop using the Trains fileserver, but now I realize you might just want to move the storage away from your local file system 🙂
Basically you'll need to send a PORT request to ElasticSearch - I'll send an example here soon 🙂
No, because the UI will still try to access http://<server-ip>:8081/... for old debug images - it won't try to access the S3
Thanks for the reply,
I saw that it prefer to change the fileserver in trains.conf to s3://XXX
So, I changed this as I wrote before.
CooperativeFox72 Logs and graphs are not stored on the fileserver - only models/artifacts (if uploaded there) and debug images (by default). For both, you have files on the fileserver (which can be moved on the file system) but you also have the URL itself registered in Elastic and Mongo, so you can't just move the file, since the UI and Trains won't find them if they try to access them using the URL stored in the metadata - you'll have to edit the metadata for that...
So for the UI each user will need to add the credentials only once, the users should have the credentials in ~trains.conf file. They just need to copy those to the profile page
Hi CooperativeFox72 ,
basically each user should do it once, just to add credentials in the user’s profile page.
About credentials for the server, let me check that for you
Thanks I just want to avoid giving the credentials to every user.
If it won't possible, I will do it..
If I will mount the S3 bucket to the trains-server and link the mount to /opt/trains/data/fileserver does it will work?
like that I won't need to change metadata
Hi CooperativeFox72 , moving old data to a different storage would require internal editing of the URL metadata registered in the events already stored in the system... It's possible, but not trivial. If you'd like, I can explain the basic principle... We're talking about debug images, right?
Hi SuccessfulKoala55 , yes for now I will like to start moving what inside the /opt/trains/data/fileserver..
because as I understand the logs and graphs are saved in elastic so I think it will not be easy to move them as well right?
Regarding of moving the fileserver to S3, what is the best way to move the old data to S3 ?
I think if I will move all the /opt/trains/data/fileserver to s3,
the trains-server will not know that right?