twilight-rs/twilight

Powerful, flexible, and scalable ecosystem of Rust libraries for the Discord API.

twilight's Issues

[gateway] Support for the new gateway version

There is now a new version of the gateway but it includes mostly voice related changes and since we dont support voice in dawn. We can ignore it for now relevant changes

[gateway] Opt-in to receive raw payloads

In the gateway, add a variant of the Event enum which contains the raw bytes of the payload. By default, this should be disabled and opt-in through the existing Cluster::some_events and Shard::some_events methods. This isn't costly because we don't consume the bytes when deserializing into a GatewayEvent, so no cloning is needed here.

[http] Expose raw client requests

Expose the functionality in the HTTP client to perform raw requests, still backed by the ratelimiter but avoiding convenience methods (such as Client::create_message).

[model] Add new field to Webhook

There is now a new a field in webhook structure called type that describes the type of the webhook and has two possible values 1, 2 which is Incoming or Channel Follower relevant commit

[cache/in-memory] Implement missing cache updates

Implement the last event cache updates for the InMemoryCache:

• presence update
• reaction remove (#67)
• reaction remove all (#67)
• voice state update

Add structs for audit log change instead of using value

[command-parser] Command parser is not deterministic and may chose wrong commands.

So at the moment the code below will sometimes fail and other times pass, it depends on the order
of the elements in the HashSet Commands and as that order is not deterministic it will sometime fail.

There are a couple of ways we could resolve this:

• Force a separator after commands
• Use a ordered set like a vector instead, but this will not solve that some commands would shadow others.

fn double_config() -> Parser<'static> {
    let mut config = Config::new();
    config.add_prefix("!");
    config.add_command("echo");
    config.add_command("echolocate");
    
    Parser::new(config)
}

#[test]
fn test_double_config() {
    let parser = double_config();
    
    match parser.parse("!echolocate") {
        Some(Command { name: "echo", .. }) => {
            panic!("Wrong command")
        },
        Some(Command { name: "echolocate", .. }) => {
        },
        // Ignore all other commands.
        Some(_) => panic!("This should not happen (1)"),
        None => panic!("This should not happen (2)"),
    }
}

[http] add 'reason' to all moderation requests

Add the 'X-Audit-Log-Reason` header to all moderation requests that can be audited.

This is already in the CreateBan request, but as a query parameter: https://github.com/dawn-rs/dawn/blob/ddfe693cb112ed8ceebb83af17ae02067bc6ab17/http/src/request/create_ban.rs#L35

We'll probably need to add better handling of UTF-8. A description of reasons is at the top of this page: https://discordapp.com/developers/docs/resources/audit-log#audit-logs

What this mostly means is manually creating the Request in the start method of requests and specifying the ratelimit header in the header map: https://github.com/dawn-rs/dawn/blob/ddfe693cb112ed8ceebb83af17ae02067bc6ab17/http/src/request/create_ban.rs#L42

[project] add 'audio'

Add the audio crate, named twilight-audio.

This is up for grabs.

Create twilight-voice

Add voice support to twilight

Add examples and docs to Test GH Action

While examples shouldn't be used in place of tests, this will help ensure docs and examples are kept in line with changes.

[model] Add message_id to optional audit entry info

Audit entries now includes and optional message_id. commit

[http] Parse error messages

Parse the error messages that Discord provides into error structs. An example of an error struct is this one, sent when a message embed's field name's length is 0:

{
    "code": 50035,
    "errors": {
        "embed": {
            "fields": {
                "0": {
                    "name": {
                        "_errors": [
                            {
                                "code": "BASE_TYPE_REQUIRED",
                                "message": "This field is required"
                            }
                        ]
                    }
                }
            }
        }
    },
    "message": "Invalid Form Body"
}

[gateway] Fix reconnection logic

On reconnects we're creating a new session but aren't propagating it to the heartbeater, so a heartbeat doesn't seem to be sent out, which then causes a close shortly after.

[ci] Don't error on decreased code coverage

We need to figure out how to make the grcov/coveralls lint not fail when coverage goes down. Right now it's disabled.

[gateway, model] Update Streaming activity type

The Streaming type now includes more data relevant commit
example payload:

{
  "details": "24H RL Stream for Charity",
  "state": "Rocket League",
  "name": "Twitch",
  "type": 1,
  "url": "https://www.twitch.tv/discordapp"
}

[gateway] Add ETF support

Add ETF support:
https://discordapp.com/developers/docs/topics/gateway#encoding-and-compression

[command-parser] Add config to have commands be case-insesitive

[gateway] Add new gateway event

The reconnect event is dispatched when a client should reconnect to the gateway (and resume their existing session, if they have one). This event usually occurs during deploys to migrate sessions gracefully off old hosts.

relevant commit

[gateway] Make compression optional

This is the first thing we need to do before getting on #14 since discord explicitly says

When using ETF, the client must not send compressed messages to the server.

Every other resume fails

Every other resume fails because the id fails to be set correctly after reconnecting to a new session.

[model, gateway] Add optional emoji field to Activity object and the new activity type

With the addition of allowing custom statuses. There is now an optional emoji field (here) and also another type of an activity.

[model] Write tests against real world payloads

Some of the models already have tests for testing against (de)serializing against real world payloads, like this one:
https://github.com/dawn-rs/dawn/blob/ad3c7c16440002d1a4b3dd7ba52066f73c392ee7/model/src/channel/reaction_type.rs#L19-L50

Tests should be added for the rest of the payloads as we can.

Add support for other runtimes than Tokio

This is a issue for adding support for adding support for other runtimes than Tokio, like for example async-std.

[ci] run cargo audit daily

Create a GitHub Action to run cargo audit daily, so that if there is a security vulnerability it can be reported here.

Update documentation across Readme's

Right now, there are enough discrepancies where I'm having to read the source code to get even a basic ping-pong bot setup.

[gateway] Request gateway URI via HTTP

On every shard connection, request the gateway URI via the given HTTP client in the config rather than hardcoding it, which is discouraged behaviour.

Command configuration: global with overrides

It'd be great to have most, if not all, options in the CommandBuilder on the parser's Config, having the options set per-command override those on the parser.

[http] Add multipart support for message creates

https://discordapp.com/developers/docs/resources/channel#create-message

Actually update cache for all events that have impl for the UpdateCache trait.

Some cache events don't actually do anything currently, an example of this is the event BanAdd, where all it does is return Ok(()).

Related is #25

[model, gateway] Add member to reaction add event

Reaction add event now includes a member field if that event happened in a guild commit

[http] Return request structs for all client methods

Change the HTTP client to return finalizer request structs for all methods. This will allow us to add new optional discord API features and our own features to requests without breaking their signatures.

See the CreateBan request finalizer, which is returned from Client::create_ban: https://github.com/dawn-rs/dawn/blob/ad3c7c16440002d1a4b3dd7ba52066f73c392ee7/http/src/request/create_ban.rs

[project] Add 'oauth2' support

Documentation here: https://discordapp.com/developers/docs/topics/oauth2

We don't need an HTTP client here, just models of the payloads, URL builders (like for building an authorization code), and maybe a stateful struct representing the flow that can be advanced.

Maybe we just want to wrap over https://docs.rs/oauth2/3.0.0-alpha.4/oauth2/? I don't know, pros and cons either way

[gateway, http] Add serde-ignored feature

Add an optional opt-in feature for the gateway and http crates to use serde_ignored to deserialize payloads and log to INFO if there are any keys we didn't catch.

This should also log unknown keys directly in the custom deserializers where applicable.

[gateway] Implement Cluster events

The stream returned by Cluster::events and Cluster::some_events is currently unimplemented. We basically just need to merge all of the streams dynamically from all of the shards as new shard sessions are up.

[model] Add custom statuses

Add support for activity type 4, custom statuses.

The payload looks like this:

{
    "t": "PRESENCE_UPDATE",
    "s": 23,
    "op": 0,
    "d": {
        "user": {
            "id": "114941315417899012"
        },
        "status": "dnd",
        "roles": [
            "620984892410429470",
            "635845304947703808"
            "621037383416283137",
            "621124153361170452",
            "621273850138591262"
        ],
        "guild_id": "620980184606048276",
        "game": {
            "type": 4,
            "state": "Rainy",
            "name": "Custom Status",
            "id": "custom",
            "emoji": {
                "name": "🌧"
            },
            "created_at": 1572196553812
        },
        "client_status": {
            "desktop": "dnd"
        },
        "activities": [
            {
                "type": 4,
                "state": "Rainy",
                "name": "Custom Status",
                "id": "custom",
                "emoji": {
                    "name": "🌧"
                },
                "created_at": 1572196553812
            }
        ]
    }
}

It looks like we just need to add the activity.emoji key - which has the same payload as channel::ReactionType::Custom - and add a test for it like with other models.

See discord/discord-api-docs#1162

ShardConfig::builder issues with queue configuration function

This could be a simple mistake on my part or something counter-intuitive but looking at the function I should be able to just provide something simple like Box::new(Queue::new()). Except this doesn't work.

A bare bones example is importing dawn-gateway and only dawn-gateway like so dawn-gateway = { git = "https://github.com/dawn-rs/dawn" } then providing the built-in LocalQueue from dawn as a reproductive example.

use dawn_gateway::{Shard, ShardConfig, queue::LocalQueue};

let token = env::var("DISCORD_TOKEN")?;
let config = ShardConfig::builder(&token);
config.queue(Box::new(LocalQueue::new()));

I would assume this would work and be how it's performed as per this line doing this exact thing, just outside the function https://github.com/dawn-rs/dawn/blob/cc35c8532171ab1e02f7a5cf664599e50465107f/gateway/src/shard/config.rs#L105

[gateway] Request from queue on connects

When attempting to start a connection, a Shard needs to go through the queue already given to it, to ensure that only one shard is attempting to connect in a given 5 second period.

[project] add 'gateway'

[gateway,model] Add presence field to `Request Guild Members` and associated payloads.

cf. discord/discord-api-docs#1116

Make EventType & Event serializable through serde

EventType & Event currently can't be serialized with serde and if you want to pass the event around to another process you currently can't.

[http] Add Path parsing

Add parsing for http::routing::Path to parse from URLs or URI paths. For example, the string channels/123/messages should parse to the path Path::ChannelsIdMessages(123).

[project] add 'cache/in-memory'

Add the cache/in-memory crate, named dawn-cache-inmemory. This is an in-memory, (mostly) immutable cache with no public locking.

Finish the implementation of `Cluster`

[http] Validate request parameters

Validate that request parameters and body values are within the accepted ranges. For example, getting some members from a guild can only get at most 1000 at a time. This will help prevent as many 400s as we can.

[model] Add new audit log events

commit

[model] member::guild_id can be non-optional

The model::guild::Member::guild_id structfield doesn't have to be optional.

Currently most types in the model crate are naive serde (de)serialization derives, and this type isn't any different. The gateway provides members with the guild_id, but the HTTP API doesn't. Since we already know the guild ID when using the HTTP API (we need it as a reference for where to get members from), we can use it as a serde::de::DeserializeSeed seed implementation for Member, avoiding the need for the structfield to be optional.

Right now, we get unfortunate real-world code like this:

use anyhow::Result;
use crate::State;
use dawn::model::gateway::payload::MemberAdd;

pub async fn handler(state: State, event: MemberAdd) -> Result<()> {
    // An unfortunate consequence of combining the HTTP and gateway models for
    // the Member struct -- we should change this.
    if let Some(guild_id) = event.guild_id {
        // work with the guild id here
    }

    Ok(())
}

Naming consistency across Config structs

ShardConfig builds into a Config, but then you start with a Config in the Command module. I would suggest explicit XConfig and XConfigBuilder names across the board, but being consistent in scheme is a minimum I would ask for in this issue.

[project] add 'cache/base'

Add the cache/base library, named dawn-cache. This is a library exporting the types from dawn-cache-trait (Cache and CacheUpdate), as well as exporting the dawn-cache-inmemory library.

Blocks on #5.

[meta] Standardize logging scheme

Standardize logging to use a single structure for messages. Right now it's stuff like "Error connecting: {:?}", "[SocketForwarder] Starting driving loop", and with #36 "[ShardProcessor {:?}] message" with {:?} being the shard ID/total.

I think the last one is the best.