rpcs3/discord-bot

Source code for RPCS3's Discord bot.

RPCS3 Compatibility Bot

This is a tech support / moderation / crowd entertainment bot for the RPCS3 discord server.

You can read the design and implementation notes by visiting the folders in the web interface, or from the architecture overview notes.

Development Requirements

• .NET 8.0 SDK or newer
• Any text editor, but here are some recommends:
  • Visual Studio (Windows and Mac only, has free Community edition)
  • Visual Studio Code (cross-platform, free)
  • JetBrains Rider (cross-platform)

Runtime Requirements

• .NET 8.0 SDK or newer to run from sources
  • bot needs dotnet command to be available (i.e. alias for the Snap package)
• Optionally Google API credentials to access Google Drive:
  • Create new project in the Google Cloud Resource Manager
  • Select the project and enable Google Drive API
  • Open API & Services Credentials
  • Create new credentials:
    • Service account credentials
    • New service account
      • if you select an existing account, new credentials will be generated in addition to previous any ones
    • Role Project > Viewer
    • Key type JSON
    • Create will generate a configuration file
  • Save said configuration file as credentials.json in user secrets folder
    • e.g on Linux this will be ~/.microsoft/usersecrets/c2e6548b-b215-4a18-a010-958ef294b310/credentials.json

How to Build

• Change configuration for test server in CompatBot/Properties/launchSettings.json
• Note that token could be set in the settings or supplied as a launch argument (higher priority)
• If you've changed the database model, add a migration
  • $ dotnet tool install --global dotnet-ef (if you have never installed the tools before)
  • $ cd CompatBot
  • $ dotnet ef migrations add -c [BotDb|ThumbnailDb] MigrationName
  • $ cd ..
• $ cd CompatBot
• $ dotnet run [token]

How to Run in Production

Running from source

• Change configuration if needed (probably just token):
  • use $ dotnet user-secrets set Token <your_token_here>
  • for available configuration variables, see Config.cs
• Put bot.db in CompatBot/ if you have one
• $ cd CompatBot
• $ dotnet run -c Release

Running with Docker

• Official image is hosted on Docker Hub.
• You should pull images tagged with release-latest (same thing as latest)
• Please take a look at the docker-compose.yml for required configuration (bot token and mounting points for persistent data).

External resources that need manual updates

• Unicode Confusables, for Homoglyph checks
• Windows Error Codes, for error decoding on non-Windows host
• Optionally pool of names (one name per line), files named as names_<category>.txt

discord-bot's Issues

Abuse Rich Presence for basic information displayment

Recently noticed how Rich Presence is sometimes abused with bots, so they can display the default help command and a link to their site / repository:

e.g. Use !help for more information. | https://github.com/RPCS3/discord-bot/

!analyze command to reanalize the log

Might be useful in case it failed to trigger automatically, or if you want to check the log again after adding a new piracy filter.
Should accept message id and/or message link, similar to !report.
Should be limited to bot mods to prevent spam and abuse.

Things to fix

• new log detections
  • if some logging was disabled
  • fw version (see RPCS3/rpcs3#5666)
  • note for phenom/fx cpus
  • add author of the log in embed author
• !!!!! interpreted as a command
• move antipiracy commands to sudo, or make separate check for !warn

Regrouping a few commands

As I was writing #180, I noticed that !psn check updates XXXXYYYYY is terribly long. There are a few ways of going about this, but first and foremost I think regrouping the psn subcommands should be the way to go.

The current hierarchy looks like this:

psn => { psn check | psn fix | psn rescan } => { psn check updates | psn check content }

As you can see, at the third level the commands are pretty damn long, and this isn't even with the parameters they recieve. There's numerous ways, as always, you can regroup this, but here's mine:

First, I don't think specifying psn really adds that much of a meaning to check updates or check content. In the former case, you really don't care about where the update list is coming from, and in the latter, it doesn't even directly come from PSN but from our mirror. So, while it is important to specify psn in the cases of fix and rescan, for check it just isn't.

Therefore, it can be removed, or more accurately saying, the check subgroup can be moved out from the psn supergroup into its own, without breaking any readability considerations.

Take then the following grouping:

psn => { psn fix | psn rescan }
check => { check updates | check content }

The commands are suddenly much shorter, but still hold the same meaning, while also retaining their sentence-like feel.

Whilst thinking about more reasons to recreate the check token as a standalone command group, my attention turned towards the !ird command. You for sure check for an .ird, and it is pretty fitting all things considered. And so while we're at it, I also recommend moving the ird command group below the check group as follows:

psn => { psn fix | psn rescan }
check => { check updates | check content | check ird }

This will need all the related help and explain commands to be updated too, of course.

#help: Assist AI

As talked about in #testers recently :P

Roadmap:

• Discuss the extent of help that the help assist should target to provide,
• Research the tech needed for implementing it (NLP, how other conversational bots work, etc.),
• Implement question difficulty gauging, so we could finetune what the bot should be able to answer / attempt to answer,
• Gather a base dataset of questions to enable testing the followup features,
• Implement basic questions/replies,
• Implement some context-aware filler questions/replies,
• Implement more advanced questions/replies (dealing with specific settings, missing LLE modules, outdated drivers, checking for an issue already existing and being reported at our github/forums),
• Letting it go in on #help as a separate user, masked as a real user instead of a bot,
• Never visiting #help ever again.

Reduce moderation spam for suicidal users

Some users get really angry and try to do as much damage as they can before getting kicked/banned.
This includes abusing bot. The most obvious feature to exploit is auto moderation with warning list.

Here's the list of things that could be done to reduce the impact:

• Limit warn list to X most recent in public channels
• Stop listing warnings after 3-5, in a 5-10 minute span
  • (?) put the user on The List of Bad Children™
• Voting for reports: auto delete message if reporter+2 other bot mods vote YES, or at least one server mod
• (?) Delete all user messages containing even a hint of a link if he is on The List

Answer questions addressed to bot directly in channels

When someone is tagging the bot with the natural question, handle it and redirect to !8ball / !rate accordingly.

Basic support for messages starting with do [you], are [you], am, what, how, etc.
Either through general catch-all event/command hook (preferably), or as a bunch of hidden commands (ugh).

Support logs compressed with RAR

Archive headers are not compressed by default and could be peeked at, similarly to zip.
Needs investigation of managed library for decompression, that is compatible with .net standard.

Docker integration

There's a desire to run the bot as a docker container, but it is also much more convenient to run and debug the bot as is, so it should be optional.

Things to consider:

• Building an image
  • Persisting configuration, if possible make it interoperable with normal bot configuration
  • Persisting logs
• CI/CD using Azure Pipelines (already has a config), AppVeyor, or Travis
• Image publishing
• Auto update integration
  • Checking for update
  • Persisting state before restart

Thumbs scraper for JP titles

Japanese side of playstation.net has a full list of all disc releases. For example: https://www.jp.playstation.com/software/title/bljs10324.html

The issue is that game page layout is not consistent, with html and thumbnail links varying wildly from title to title.

• investigate the feasibility, at least for most common layouts
• client for scraping / extraction
• plumbing to add it to the current scraping pipeline

Censorship filter

Similar to the piracy filter for user messages, but to no warnings, and optional custom explanation of why it was removed.

• db model update
• new filtering handler
• bot management commands
  • add new filters
  • list the filters
  • remove the filters
  • update explanations

#media moderation using OCR

People keep posting screenshots of pirated games. It would be cool to OCR window titles and check piracy filters on them.

• investigate availability of OCR libraries for dotnet core
  • IronOCR licensing is not clear for OSS
  • teserract wrapper has some native dependencies
  • Azure Cognitive Services priced at $1.5 per 1000 request batch, up to 1M requests per month
• optionally detect window title to crop before OCR

[Request] Limit unspecific !pr to bot-spam or dm

If using !pr with nothing after it'll pull up every open PR with links. Which is fine, but it seriously spams the channel it is used in.
Similar to !explain list, it would be better off not used in main channels. While of course allowing specific PRs to be linked still.

!e3 countdown command

Hey, some users (me and @MsDarkLow mainly) would really appreciate some sort of E3 countdown that could be requested from bot and since I suck at GIT, I am not able to open a PR for this. Here is a working code that I wrote (change it to your liking). It would be awesome if someone would implement this functionality. No configuration is needed since E3 is always (let's hope it stays that way too) at the same time every year, so I made it future-proof.

    //E3 usually starts with MS at 20:00 utc
    msg = "";
    DateTime e3 = new DateTime((DateTime.Now.Month >= 6 && DateTime.Now.Day > 11) ? DateTime.Now.Year + 1 : DateTime.Now.Year, 6, 11, 20, 0, 0).ToUniversalTime();
    TimeSpan e3left = e3.Subtract(DateTime.Now.ToUniversalTime());

    if ((DateTime.Now.ToUniversalTime().DayOfYear > 162) && (DateTime.Now.ToUniversalTime().DayOfYear <= 166))
    {
        msg = "E3 " + DateTime.Now.ToUniversalTime().Year + " is happening right now!";
    }
    else
    {
        msg = "E3 " + e3.Year + " starts with Microsoft's conference on " + e3.ToShortDateString()
            + Environment.NewLine + "Time to E3: " + ((e3left.Days > 0) ? e3left.Days + " days, " : "") + e3left.Hours + " hours, " + e3left.Minutes + " minutes left.";
    }

Reduce spam for name spoofing log

In response to the jarves incident, spoofing alerts need to be tweaked.

I'm thinking it's useless to do further reports once the potential target list grows past 2, as it's probably just another one of those meme strikes.

[Feature Request] `!psn check updates` Improvements

Example usage: !psn check updates ni no kuni

In order to avoid having to implement a whole search / fuzzy search functionality over the metadata db, this should / could be done via asking 🅱️Ani's api for compat list in regards to the query string, extract the first matching titleID, and pipe it to the currently used function.

Maybe the bot should also echo some info onto discord along with the found updates, like:
Showing updates for Ni no Kuni - Wrath of the White Witch [BLUS30947].
If this isn't the list of updates you were looking for, please use a titleID instead.

When implemented, the explanation of the command should also be updated with the freshly added syntax.

Update: I'm a dumdum, and forgot that there are multiple titleIDs for each game smh. I should go back to rpcs3 school quick.

Anyways, this would also entail that if you use a game title as your query, you'd get results for all the regions and mediatypes the given game has been released in. This does mean that the resulting list will be pretty long, though I'm fairly sure the command is already limited to #bot-spam, making this less of an issue.

Auto-limiting to one region & media format could maybe applied when the list is too big. (see Gran Turismo games)

Extra specifiers could maybe added as well, not sure how to go about them. They could help filtering the possible titleIDs, by forcing a region (e,g. eu, usa, etc.) and optionally a mediatype (disc/digital).

Example: !psn check updates eu disc tlou.

All in all, needs a bit more brainstorming, but I think the general idea is definitely nice, and more galaxybrain user-friendly too.

!explain dump as a plain text

New subcommand for !explain to export the explanation with preserved formatting as a plain text attachment.

• version that accepts the term as an argument
• version that accepts any message (e.g. as a link)

!rate my/yourself/etc

Implement more natural language processing for pronouns. Replace with user's / bot's ids similar to other special cases.

Add an ability to put specific users on bot's ignore list

Such that bot would ignore any message from said users. Should be useful for repeated offenders and bot abusers.

Add a note for forced af when the value is not 16x

As all other options are pretty much pointless for overriding

Collect message box texts maybe?

Could be useful for non-ascii messages that are not rendered in Native UI at the moment.

In logs it looks like this:

·W 0:01:44.000717 {PPU[0x1000000] Thread (main_thread) [0x013c51a4]} cellSysutil: cellMsgDialogOpen2(type=0x87, msgString=“ダウンロードコンテンツをチェック中です
”, callback=*0x11b75d8, userData=*0x0, extParam=*0x0)
·W 0:01:44.000721 {PPU[0x1000000] Thread (main_thread) [0x013c51a4]} cellSysutil: ダウンロードコンテンツをチェック中です

Interactive greeting for new users

This is mostly to prevent people from ignoring reading the rules and other important stuff.

The preliminary flow might look like this:

• on join, give new user a role that prevents them from doing anything except for reading the rules channel
• when the time is up, do an interactive quiz of ~3 questions (i.e. ask for help in #help, do not ask for pirated content, provide logs when appropriate, etc)
• if answered correctly, remove the inmate role

Needs more research on how permissions can be used to full effect.

Replace custom GitHub client with official GraphQL one

Right now we're using a tiny custom-built client that is using older REST API version. It would be nice to migrate to a properly supported official client. This would also help in the future if we'll have to use authentication for increased request limits.

Note when user has inadquate cpu in the log results

• less than 4 threads
• fx and phenom cpus
• (?) ultraportable intel cpus

alternately, rely on #108 results

Auto-warn for piracy in case the pirate site's name is visible via Discord Rich Presence

Title says it all, some dude came in to the server today with a pirate copy of P5, and Discord Rich Presence had the name of the pirate site displayed while he was playing lol

XAudio2 issues diagnosis

XAudio2Thread : CreateMasteringVoice() failed(0x80070490)

this indicates that no audio output devices available, bot can mention this in bot log results

RPCS3.zip

IRD lookup command

A command similar to game updates, but for ird lookup.
IRD Library as a source, it has json api that is used for the website itself.

• reverse engineer api
• create client for the api
  • ird download link
  • optional metadata information

Support logs compressed with 7z

This one is ugh. Everything is compressed. No documentation on file format. Official SDK only has container handling code in C++ implementation (certainly not in C# project).
Needs a lot of digging and assessing the feasibility.

ToDo Items

This is just to track what needs to be added in the next update.

• AMD drivers after 19.12+ break OpenGL
• Sly Cooper 4 resolution scaling requires CPU Blit enabled, also needs motion controls
• Make disabled TSX an info instead of a warn
• MSAA warning for 5500/5700 GPUs
• slow cpu warning for all 4 core cpus without TSX
• windows 10 20h2 builds check

Proper certificate checks for the game update api

Right now all certificate checks to the psn update api are ignored (see CustomTlsCertificatesHandler.cs).

I don't really like this and do want to implement a proper certificate chain validation. Unfortunately, root certificates are not public, and only redistributed through ps3 firmware update. On top of that, currently there's no way to implement dynamic chain trust check with custom root certificates via official api. The only viable solution is to check and import root certificates to the user personal certificate store on startup.

Needs more investigation.

Add FilterAction to supress reports in bot log channel

This should be useful for filters to autoremove [object Object] messages from Yappy

!pr command group

• !pr num to link the specified pr on github (embed with title and description)
• !pr build num to link the latest appveyor build for the specified pr
• !pr list to link currently open PRs

!explain shouldn't be case-sensitive

The easiest way is to normalize terms to lowercase on add/explain.
Doing case-insensitive db schema update isn't as easy, and fraught with frail (both in Unicode support and performance).

Integrate Ink scripting

Ink is a simple yet powerful text-oriented scripting language that works great for choose-your-own-adventure type of games, but also for any conversation tree management.

It can be used to make some fun stuff like interactive greeting (#103), quizzing (!jail command), interactive troubleshooting, etc.

Requires proper .net standard nuget package first tho (see inkle/ink#515)

!meme command

Current implementation design looks like this:
!meme command is marked as hidden, and is not listed in the !help command list.
Any user can add a meme bound to their user id, in a similar vein to !explain command. Limited to 5 (?) per user.
Then this user (and this user alone) can use !explain to show the content of the meme.
Bot automatically adds a 👍 reaction to that message, and will count the number of reactions through the even handler. Everything is summed up to produce a total score.
The top 5 of all memes will be featured in the general !explain list and could be shown/viewed by any user, not just by the meme author.

This will require db model update and a lot of plumbing between the tables and handlers.

Command disable feature

Need a command disable feature only available to bot admins.

Make a better !help module

Default implementation is lacking and very confusing for regular people to understand how to use.
Need to think how to better instruct in layman's terms.
At the very least there should be examples, and maybe a full list with arguments and short descriptions, like in old bot.

Autosync bot mod list with discord roles

Developers and up should have bot mod rights, but it should still allow for manual additions.

• sync on bot start
• sync on member role change
• track if it was synced automatically
• remove from the list on member role change if it was added automatically

[Feature Request] Let compatbot highlight if the user uses an outdated build

The idea is that when a user uploads their log, the compatbot would use some visual clues to highlight to both the user and the helpers, that an outdated RPCS3 build is being used.

The bot doesn't actually have to check if the build is outdated or not (since there's no proper build numbering), only if the build number matches with the one found in something like the !latest.

Add more replies to 8ball / rate commands

Let's make this bot as great as it could ever be.
The only requirement is to keep the balance of positive/neutral/negative connotations, erring on the side of being more positive.

Current lists can be found here: Misc.cs

Please keep these points in mind:

• Try to keep slight spelling variations to a minimum, we need quality diverse responses
• Try to keep positive-to-neutral-to-negative ratio around 3:2:3 to 3:2:2
• Please update response counters in the comments accordingly, for easier tracking

Cross-reference 'sys_fs_open' failed with the ird

IRD file contains all file names that are expected to be on the disc, it might be helpful to auto detect broken copies.

• ird fetch (depends on #89)
• local ird cache
  • try to get matches via web api, use local copy when possible
  • use local cache search if web is not available
• ird metadata decoder
• new log parser collector for 'sys_fs_open' failed messages
• same for 'sys_fs_opendir' failed messages
• formatter for the Notes section of log parsing results

'sys_fs_opendir' failed.log.gz
'sys_fs_open' failed.log.gz

Issues with bot reaction matching

Some fuzzy matching rules seem to be applied to areas that really don't need it, leading to oddities. In particular, the bot will assume it is being discussed if anyone uses any word beginning with "bot", regardless of the word itself, including "bothered" or even "botasdf".

Test case "great botasdf"

In any case, this really just highlights that there should only be a subset of keywords to which fuzzy matching is applied. Cheers m8.

Show prompt for unsupported log archive types

Bot should prompt to upload .zip version for rpcs3.7z/.rar attachments

Detect errors with /dev_flash/

Should probably look for all error lines that have /dev_flash/ in it

can't find certificate.zip
typical 'no firmware'.zip

!top without arguments crashes the bot

Culprit:

discord-bot/bot.py

Line 94 in e9a00b9

if args[0] not in ("new", "old"):

Put it after the argument count check and you're good

!rate cpu command

This can give you an approximate suitability of the given cpu model for the purpose of ps3 emulation.

• research the means of getting the basic cpu info by its name (core topology, base clocks, extensions)
• implement guestimates for the metrics
• do formatting with emoji scale just for fun (?)

[Idea/Entertainment] Rate command

A command for entertainment just like !8ball

Add a command "rate", example:

• Me: !rate loweys
• Bot: Baaaaaaaaaaaaaaaad

The bot will randomly rate anything you have written in front of it.

With a bonus if someone rates something specifically, you'll get a specific answer too like
"!rate Nekotekina" "Russian Kot is supreme amazing"

Idunno. Take your time™
May Yakuza guide thee

Stats for top explain / serial lookup / piracy

A very miscellaneous task to implement tracking of various usage stats.
For example:

• usage numbers for !explain terms
• popular games that are looked up through serial numbers / !compat queries
• number of piracy triggers
  • total
  • per day
  • days since last trigger
• bot command usage
  • per command
  • per command group

Replace PSN scraping with tmdb api calls

See wiki for how to generate links to the metadata.

An ability to force username changes by admins

Some people are really annoying and try to change their usernames to break the rule about the diacritics vomit. Usually they get a kick, but sometimes they return, or mods are unavailable.

It would be useful to keep a list of such users and force the specified username at all times.