payload-bot/payload-neo

The largest TF2-based Discord bot serving over 50,000 users daily.

payload-neo's Issues

Convert to slash commands

Move API outside of repo

idk use grpc or something cool to query client

[Feature] Sharding

Nearing the 1000 server mark, probably time to think about sharding.

Rewrite pushcart to sql

Because we can do that. It'll be much easier to rollback changes + only 1 query :)

Convert scheduled scripts to use new command structure

Convert them to use classes, the abstract class is just not even used at all.

[Chore] Move all config.ts items that are keys to ENV

This ensures that everything can be done in the ENV, and never be overridden.

Also should invest in obtaining a new AWS api key.

Port sourcequery to gamedig

use gamedig instead of sourcequery.

[Feature] Turn the pushcart timeout into a scheduled script

Right now, payload currently checks to see if the person is over the limit, and if so, the person will be restricted until 24 hours after that limit. That's stored in the database, and its only until that person trys to pushcart again that it checks again the time left.

If this timer were a script, it would eliminate that check and always be up to date. This script would need to run pretty often, and maybe a map would be better suited to this, where that map would be controlled by that script (maybe new instances are created of that timer?)

Not sure the impact of this, nor do I think this is something that needs to be done ever. Think this could be a cool way to manage the time outs of the pushcart limiter.

[Rewrite] Use i18n for localization

Makes sense to have the i18n library do a lot of the work that I would have to do manually.

Not a breaking change, but it would prove difficult to rewrite all the commands again.

[Feature] RGL / UGC "playercheck"

Command: playercheck

Takes in a server information (rcon string) and retrieves all user's from that server. Probably best suited for a website.

Memory Error

Memory runs out of the VM every ~5 days or so.

Snipe cache only runs if a message has been deleted or updated. Needs to be moved to a scheduled command.

TFTV auto response uses server time

bug- date is wrong cus server time lool

instead of

RGL commands

Add pls rgl, rgl team and match previews

[Feature] Automatic posts of TF2 updates

Like patchbot, except would be in Payload.

Could reduce # of bots in a given discord, but also would fit the theme perfectly.

[Feature] Payload tells you time left when you reach your point limit [pushcart]

Right now, it tells you to "come back tomorrow".

The user's expiration date is stored in the database, compare that to the current date and use moment to convert that to a usable datestring.

Would make it super nice and clean and awesome to use, rather than have peope guess the time remaining, only to be disappointed.

Convert to Sapphire V3

Because we want some more flexibility

Feature: port to esbuild/swc

11 seconds to build a bot is outrageous. Use swc or esbuild

https://github.com/sapphiredev/examples/tree/main/examples/with-tsup
https://github.com/sapphiredev/examples/tree/main/examples/with-swc

swc cooler though 👀

Use Github Actions

Github actions with webhook postings so we have visual indication of deployment status and maybe even tests / building. Would prevent bot from being updated when something can't build.

Redo command restrictions

Make them work as modules instead, meaning removing per-channel restrictions to full on guild-wide.

This will probably make people mad.

[Rewrite] Remove sourcequery in favor of other rcon packages

Sourcequery doesn't have types, but we have a package that has types and is reliable in production. Let's switch to that package so we can remove that and type our server.execute command.

Port to SQL

SQL has a lot of advantages over mongodb. I store a lot of relational data. Let's put it in a relational database.

1. Update to use Prisma
2. Convert current contracts into a more friendly version
3. Update all connections
4. Find out how to get auto-migrations working (that should be easy)
5. Create a script or run during downtime

Probably want to go with postgres or sqlite. Sqlite might be the best option here. Investigate which option would be best for deployment and testing.

[Rewrite] API Changes

Change all API routes to use /api/v1/(internal | external)/ (route)

For example, our request to GET /api/stats now => GET /api/v1/internal/stats
Or, POST /api/rcon => POST /api/v1/external/rcon

This rewrite would make use of the express router, so handling routes would be better eased. Leaves a ton of room in the index.ts file to be for the dashboard site.

A "catch-all" route would be applied as well, and would tell the user that it was not found on the server. This will come in handy for the dashboard site I believe.

All external routes would be revamped as well perhaps, as the rcon uses a promise which is not applicable there.

Pushcart rank shows 1 even if you're not top dog

prefix changing from dashboard doesn't reflect on bot immediately

Describe the bug
prefix from dashboard doesn't change instantly

To Reproduce
Steps to reproduce the behavior:

1. go to dashboard
2. change prefix
3. try new prefix

Expected behavior
prefix to be changed

Additional context
cache issue when getting the prefix from database. Flaw in design currently.

fixing it requires the cache to be bypassed. probably not a bad thinng anyways as the cache kinda... sucks.

Remove nest

Remove Nest
There's some issues regarding nestjs that isn't easily solvable.

Problem 1: Fastify
Fasify would make this API much faster and use slightly less resources. The problem: Passport. We heavily rely on passport for authorization, so this makes it not viable without major reshuffling or using another package (notably fasitfy-passport)

Problem 2: Speed
We duplicate a lot of code. Serialization, contracts and container are all repeated. This results in a lot more connections to mongodb and some heavy packages. It also prevents us from using swc or tsup in #135.

Problem 3: Unified structure
There's a lot more going on with nest than I imagined, and having DI, while great, isn't really what is needed- it's performance. API could get hit often with a variety of things, and thus performance should be the next focus.

Overall, nest brings great DX but it suffers with the package ecosystem it has.

This also should help resolve issues with the dashboard being dirt slow.

Pushcart leaderboard should have privacy settings

We should have toggles to show or not show the current server on pushcart. This is for privacy reasons.
We could also just not care..... and have a "you use this command, you agree to the terms" but that's no fun.

Port to Fastify

Fastify is 🚀 fast. Convert from express adaptor to this new one.

Tried in #125 but failed because the middleware strategy used fails when using */ in the url for middleware consumer. I'm certain it's a bug in nest but I can't be bothered to fix it yet.

Just fork and switch to feat/fastify branch and PR to that branch. Can revisit it in the future.

feat: add metrics

Metrics:

• Command rates
• Prisma metrics
• API success rates
• Logger errors versus other stuff

Idea: caches for API

New API has most routes at ~100ms range, with most of the server configurations nearing ~500 ms.

Should probably create some sort of cache service to reduce load on database when making these (repeated) queries, as 99% of the queries are lookup for permissions, then the last ones are updating.

Issues: Cache invalidation

Pushcart doesn't display names properly

[Feature] Server Integration

Idea
There’s currently an autoresponse that allows users to post a log and Payload will render a preview of that. In this case, running a plugin on a server will allow users to automatically post logs into discord.

Integration
A new command (pls server -> change pls server to pls rcon)
This would send the user a unique ID (probably from uuid- or some other library) that would be used instead of a discord channel. This would link the channel to the unique ID, so no one could steal and abuse that (like making their own server, and grabbing a channel and sending bad stuff through the api).

A command to remove the ID would be needed, and probably limiting the bot to 1 ID per discord server would be ideal. An edit command as well would be useful, so you can edit the channel (maybe this would keep the same ID?)

Api Changes
POST /api/logs
Returns status (401, 500, 200 ect).
Potentially include the channel it was sent to, or maybe the id?
Use Authorization header with their included key to send over the network.

Feature: paginate logs command

the "to do":
paginate logs command. Wait... we can't without button handlers.

Problem: there's no way of lazy loading paginations. We'd have to do something silly. Needs research.

Webhooks do not send

Webhook URL is wrong on API. Change that or change in the plugin (do it on the bot lol)

[Chore] Port to nestjs

Express API can be converted to nestjs, and make the API so much cooler

Use an image instead of needing to build every time

Instead of making the image with npm run start, I can change that to make it npm run build, which will make an image up-to-date with everything, and dockerhub can build it in the cloud.

This makes it faster for redeployment, I only need to re-download the image, rather than taking up time to build it.

Changes needed:

Dockercompose file will need to have an image (probably payload-neo)
Dockercompose file will have to have an entry command, npm run start
Dockerfile will need to change the npm run start to npm run build

Subcommands don't register user permissions

There is no check for user permissions. This should be taken care of before running so sub commands can be registered correctly

Port to Node v14

node 12 -> 14 to handle the new lts of node.js

also, we finally can use nullish coalescing and optional chaining, god bless.