Unanswered
If The Trains-Server Stops Responding, Would Any Running Experiment Keep A Cache Of To-Be-Sent-Data, Fail The Experiment, Or Continue The Run, Skipping The Recordings Until The Server Is Back Up?
Hmm TrickyRaccoon92 take a look at the cleanup service, I think you can hack it so instead of deleting the artifacts, it will archive them somewhere (also you can change the filter, maybe only perform on experiments with specific user tag)
What do you think?
https://github.com/allegroai/trains/blob/master/examples/services/cleanup/cleanup_service.py
100 Views
0
Answers
3 years ago
one year ago