yeah, backups take much longer, and we had to increase our EC2 instance volume size twice because of these indices

got it, thanks, will try to delete older ones

If these indices tend to grow large, I think it would be cool if there was a flag that would periodically remove them. probably a lot of users aren't aware that these take up so much space

Hey DilapidatedDucks58 !
These indices are not mandatory, and basically provide historical information on worker statistics and performance that is displayed in the workers and queues page, and is also available for debugging and optimization purposes.

You can safely delete them (also periodically using a script, if you'd like). From what I see you're more concerned with their size on disk and no from the number of indices (1 each month)?

Can you do selective backup and simply not backup these indices?

AnxiousSeal95 I see what you're getting at, however this is more of an automated devops task and less of a server feature - in general, we're always much more careful when deleting stuff, and it's always advisable to have separate processes for backup and deletion that can be externally controlled, maintained and monitored.