Automatic ZIP archiving and removal of old log files

[jackdotjs]

Automatic ZIP archiving and removal of old log files

[JackDotJS]

Forgot to mention, this should also apply to the profile backups thing I mentioned here: #12 (comment)

[JackDotJS]

As suggested by @sp614x, we could zip a group of logs first, and then delete them later.

What I could do is take logs and group them by each month they were generated in. These zipped files could be moved into a separate "archive" folder, where they can then be deleted like a year later.

[JackDotJS]

Still relevant for NX. Log files pile up like crazy, at least in a development environment.

[JackDotJS]

honestly, i've been putting this off for so so long, it's unbelievable. frankly it's mostly because i dread trying to imagine how this would be programmed. doing stuff with the file system is NOT fun, but i think i've finally figured out how this could be done in a relatively simple manner:

rather than on EVERY startup, log archiving should be done at the start of each month (simply check if date === 0 during boot). at the start of each month, the bot can search through the entire logs folder and group each log file by their month and year. these files can then be archived as <month> <year>.zip (or 7z if possible). as a fail-safe, the bot will ignore all files that were created on the current month and year. additionally (probably after emptying the logs folder), the bot can go through the archive folder and deleted archives that are exactly over a year old.

as for getting the file ages/dates, i'm still considering whether this should check the actual file metadata or the file name. the file names would likely be more reliable, but they're too easy to manipulate. file metadata, on the other hand, will ALWAYS give some kind of valid date, but it is incredibly unreliable in my experience.

alternatively, i could just check both and pick the oldest date. idunno

[JackDotJS]

mostly done, still need to delete year old archives (both profiles and logs, actually)
will do this later

[JackDotJS]

added in 5bb6f87.
will keep open for a later update

[bdotsamir]

This isn't difficult, I implemented it myself, check it out: https://gitlab.com/akii0008/lwbot-rewrite/-/blob/master/src/startup.js#L110

[JackDotJS]

from #223:

the only potential issue i see with the current design is that any crash that happens right at the end of the month can only be viewed by accessing the archive manually, assuming they're being looked at on or after the first day of the next month. a potential solution to this may be to only archive files once they are over a month old, instead of archiving ALL files within a given month after that month has passed.

side note: this can probably be done with relatively minimal changes to the current script, but it will require running the script every day on startup, which can (and will) increase the overall boot/restart time.

[JackDotJS]

still todo: delete year old archives

[JackDotJS]

more todo: backup new guild configs (#285)