discordjs / voice Goto Github PK
View Code? Open in Web Editor NEWImplementation of the Discord Voice API for discord.js and other JS/TS libraries
License: Apache License 2.0
Implementation of the Discord Voice API for discord.js and other JS/TS libraries
License: Apache License 2.0
#33 implemented a link to the Contributing Guide in the README (https://github.com/discordjs/voice/blame/main/README.md#L35), though CONTRIBUTING.md does not exist.
cc: @iCrawl
These are the scenarios for a stream finishing (defined as there being no data left to read from the stream):
close
, end
or finish
are emitted). This is already handled by the library. Since no errors are emitted, eventually, the AudioPlayer#_step()
method will find that the stream is no longer readable and transition to an idle state.Handling an error should look something like this:
error
eventAdditionally, this condition is used to check whether to terminate the _step
method: state.resource.playStream.readableEnded || state.resource.playStream.destroyed
. This could really just be replaced by !state.resource.playStream.readable
which is also more readable.
When an audio player is unable to take 5 consecutive audio packets from a stream, the stream is killed. This limit should be a configurable behaviour.
It would be nice to have a comprehensive example showing how an audio player (using a queue) can be implemented.
It would be useful to include the following debug information:
VoiceConnection
, Networking
, and AudioPlayer
Currently, @discordjs/voice
has made improvements over Discord.js v11 and v12 voice by updating the audio dispatch cycle.
In v11 and v12, each audio dispatch cycle is 20ms long. At the start of the cycle, an Opus packet is read, encrypted, and dispatched. The reading (the underlying Opus encoder is thread-blocking) and encrypting of the packets can block the thread for up to a few milliseconds, which is actually quite significant for audio work. The time required for each of these two steps is also variable. For example, in one cycle, it may take 0ms to do these two things. In another, it may take 2ms.
This causes a problem where the audio packet is dispatched at times that aren't exactly 20ms. In the example above, there would be a 2ms gap between the first and second audio packets being dispatched, and this can be observed in the Discord client as a very short gap in transmission.
A fix was explored in @discordjs/voice
which involves performing packet dispatch at the start of each audio cycle. This results in a much more consistent 20ms interval. However, this only works with one audio player - adding another audio player could recreate the situation above. One audio player may be scheduled to perform its next audio cycle at the same time the thread is being blocked by packet encoding/encryption of another player.
The ideal fix would of course be to make this blocking work non-blocking. However, an additional and more realistic fix for the time being is to instead have a single clock that drives each audio player.
At the start of each interval of the clock, it should dispatch audio packets from all the audio players, and then allow each audio player to do the thread-blocking work of preparing each audio packet. After this is done, the timeout should be refreshed so that the next cycle is ready to begin 20ms after the current cycle started.
This would allow for smoother playback when more than one audio player is used.
In the future, we should definitely explore moving the blocking work to worker threads. While #52 is open, I've also found that every 30 seconds of audio playback involves blocking the thread for around 1.8 seconds with just the Opus encoder - this is far from ideal!
After a very rudimentary test, I found that 40 seconds of playback of an Ogg Opus audio stream had spent 105ms time blocking the thread with packet encryption using sodium
on my system.
This could contribute to stream stutters at scale.
Offloading this task to worker threads can help to reduce this.
Couldn't think of a good title
When using the example code as written, from my experience it would not play without waiting for the resource to be ready.
player.play(resource)
Modified line
resource.playStream.once(`readable`, () => player.play(resource));
I would submit a PR to modify the example, however I'm unsure if this is an edge case due to some unknown cause.
Further details:
0.2.0
14.17.0
Windows 10 Pro Build 19042
Relevant client options:
Using discord-rose
at master.
Currently, the biggest thread-blocker in playing audio is Opus encryption using @discordjs/opus
or opusscript
. Because the transformer pipeline is fairly independent from the rest of the library, it would be possible to offload this work to a worker thread so that the main process doesn't get blocked.
This would help with reducing jitter/stutter in streams.
It would be nice to have a comprehensive example showing how to use voice receive.
If you want to play an audio resource on an audio player, you need to wait until the stream becomes readable:
const player = createAudioPlayer();
const audioResource = createAudioResource(...);
audioResource.once('readable', () => player.play(audioResource));
Without this check, the audio player could attempt to read the stream before it is ready and eventually decide the stream is unplayable before destroying it.
Additionally, the above example doesn't even handle the case where a stream might never be readable.
This isn't really a user-friendly experience. Instead, it would be nicer to add an additional state to AudioPlayer, e.g. Preparing
or Loading
, that transitions into Playing
once the stream becomes readable. The user can then decide how long they want to allow the player to stay in the loading state before giving up and cancelling.
For usability, it would be nice if we could have default values for createAudioResource
options.
These default values could be:
inputType
: StreamType.Arbitrary
name
: unsetinlineVolume
: falseThis would allow createAudioResource('file.mp3')
instead of createAudioResource('file.mp3', { inputType: StreamType.Arbitrary })
@kyranet raised a good point in https://github.com/discordjs/guide/pull/595/files#r573217744. The library should prevent users from playing the same resource across multiple players. This could be achieved by having the resource "locked" while being played by an audio player, and then attempting to play a locked resource should throw.
If the producer of a voice adapter realises that the voice adapter can no longer be used (e.g. shard has been disconnected), it is currently unable to relay this information to @discordjs/voice.
In this situation, the producer should be able to signal to the library that the adapter can no longer be used, and the library should destroy the voice connection. When it is possible to recreate the voice adapter, the user can then recreate the voice connection.
If a user wants to create a resource and catch errors, they will have to do something like this:
const player = createAudioPlayer();
const resource = createAudioResource('input.mp3', options);
resource.playStream.on('error', console.error);
player.play(resource);
They would have to apply the error
handler for every resource they create. Maybe it would be better for these errors to be propagated through the AudioPlayer
when the resource is connected to it?
The library could easily infer some stream types, e.g. a stream would be input type OggOpus
if stream instanceof prism.opus.OggDemuxer
held. This would save from having to specify streamType so explicitly.
Currently, the NoSubscriberBehaviour
exists but it cannot be set by the user and is currently fixed to one mode.
This should be updated to allow the user to pick the behaviour that they'd like.
Currently, createAudioPlayer([options])
takes optional options, but if you specify options, then you need to specify all of the possible options.
The function signature should be updated to allow passing in partial options.
I assumed that Readable.readable
was synonymous with the readable
event, which is emitted when there is data to be read from the stream. Instead, Readable.readable
actually means that the stream has not been destroyed or emitted 'error' or 'end'.
AudioPlayer
should be updated to check for this properly.
When trying to build with tsc
, I get several missing module errors. I'm not sure if this is an edge case or not, as I'm not using discord.js
, rather a library called discord-rose
, so my suspicion is that these missing modules are installed with discord.js
, so they are overlooked. Here is the error stack I received:
node_modules/@discordjs/voice/dist/audio/AudioPlayer.d.ts:5:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
5 import TypedEmitter from 'typed-emitter';
~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/DataStore.d.ts:1:32 - error TS2307: Cannot find module 'discord-api-types/v8/gateway' or its corresponding type declarations.
1 import { GatewayOPCodes } from 'discord-api-types/v8/gateway';
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/networking/Networking.d.ts:4:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
4 import TypedEmitter from 'typed-emitter';
~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/networking/VoiceUDPSocket.d.ts:2:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
2 import TypedEmitter from 'typed-emitter';
~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/networking/VoiceWebSocket.d.ts:2:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
2 import TypedEmitter from 'typed-emitter';
~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/receive/SSRCMap.d.ts:1:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
1 import TypedEmitter from 'typed-emitter';
~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/util/adapter.d.ts:1:91 - error TS2307: Cannot find module 'discord-api-types/v8/gateway' or its corresponding type declarations.
1 import { GatewayVoiceServerUpdateDispatchData, GatewayVoiceStateUpdateDispatchData } from 'discord-api-types/v8/gateway';
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
node_modules/@discordjs/voice/dist/VoiceConnection.d.ts:8:26 - error TS2307: Cannot find module 'typed-emitter' or its corresponding type declarations.
8 import TypedEmitter from 'typed-emitter';
When configuring tsc
to ignore the errors, the library does work like normal.
Further details:
Relevant client options:
Using discord-rose
at master.
Someone in the Discord server posted this trace when trying to use the library with tweetnacl:
2021-06-09T17:27:40.526365+00:00 app[worker.1]: /app/node_modules/@discordjs/voice/dist/networking/Networking.js:413
2021-06-09T17:27:40.526401+00:00 app[worker.1]: secretbox.methods.close(opusPacket, connectionData.nonceBuffer, secretKey),
2021-06-09T17:27:40.526402+00:00 app[worker.1]: ^
2021-06-09T17:27:40.526402+00:00 app[worker.1]:
2021-06-09T17:27:40.526403+00:00 app[worker.1]: TypeError: secretbox.methods.close is not a function
2021-06-09T17:27:40.526404+00:00 app[worker.1]: at Networking.encryptOpusPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:413:35)
2021-06-09T17:27:40.526404+00:00 app[worker.1]: at Networking.createAudioPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:397:53)
2021-06-09T17:27:40.526404+00:00 app[worker.1]: at Networking.prepareAudioPacket (/app/node_modules/@discordjs/voice/dist/networking/Networking.js:321:37)
2021-06-09T17:27:40.526405+00:00 app[worker.1]: at VoiceConnection.prepareAudioPacket (/app/node_modules/@discordjs/voice/dist/VoiceConnection.js:282:33)
2021-06-09T17:27:40.526405+00:00 app[worker.1]: at /app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:419:54
2021-06-09T17:27:40.526405+00:00 app[worker.1]: at Array.forEach (<anonymous>)
2021-06-09T17:27:40.526406+00:00 app[worker.1]: at AudioPlayer._preparePacket (/app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:419:19)
2021-06-09T17:27:40.526406+00:00 app[worker.1]: at AudioPlayer._stepPrepare (/app/node_modules/@discordjs/voice/dist/audio/AudioPlayer.js:391:22)
2021-06-09T17:27:40.526406+00:00 app[worker.1]: at prepareNextAudioFrame (/app/node_modules/@discordjs/voice/dist/DataStore.js:71:31)
2021-06-09T17:27:40.526407+00:00 app[worker.1]: at audioCycleStep (/app/node_modules/@discordjs/voice/dist/DataStore.js:56:5)
This was fixed after they installed libsodium-wrappers.
Lines 35 to 37 in a124f95
Consider adding eslint-plugin-tsdoc
to a linting workflow to automate discovery of such mistakes.
The last important piece of code to be tested is the Networking.ts file. Once this is done, we can start to move towards a v1.0.0 release ๐
Should allow you to see the ping
data of a connection - similar to Discord.js's WebSocketShard
.
This should be implemented in the Networking
class. If the user wants to access the ping
stat, they should have to access it directly:
if (voiceConnection.state.status === VoiceConnectionStatus.Ready) {
const ping = voiceConnection.state.networking.ping;
}
This is to stay in line with the rest of the library - aliases and getter utilities are reduced wherever possible.
During listening, playback is abruptly interrupted with an error.
Error:
AudioPlayerError: aborted
at connResetException (node:internal/errors:683:14)
at TLSSocket.socketCloseListener (node:_http_client:407:19)
at TLSSocket.emit (node:events:377:35)
at node:net:661:12
at TCP.done (node:_tls_wrap:577:7) {
resource: AudioResource {
playbackDuration: 93900,
started: true,
edges: [ [Object], [Object], [Object] ],
playStream: Encoder {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
encoder: null,
_options: [Object],
_required: 3840,
_buffer: null,
[Symbol(kCapture)]: false,
[Symbol(kCallback)]: [Function: bound onwrite]
},
metadata: Track {
url: 'youtube url',
title: 'video title',
onStart: [Function: onStart],
onFinish: [Function: onFinish],
onError: [Function: onError]
},
volume: VolumeTransformer {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
_readInt: [Function (anonymous)],
_writeInt: [Function (anonymous)],
_bits: 16,
_bytes: 2,
_extremum: 32768,
volume: 0.5,
_chunk: null,
[Symbol(kCapture)]: false,
[Symbol(kCallback)]: [Function: bound onwrite]
},
encoder: Encoder {
_readableState: [ReadableState],
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
_writableState: [WritableState],
allowHalfOpen: true,
encoder: null,
_options: [Object],
_required: 3840,
_buffer: null,
[Symbol(kCapture)]: false,
[Symbol(kCallback)]: [Function: bound onwrite]
},
audioPlayer: AudioPlayer {
_events: [Object: null prototype],
_eventsCount: 2,
_maxListeners: undefined,
subscribers: [Array],
_state: [Object],
behaviors: [Object],
debug: [Function (anonymous)],
[Symbol(kCapture)]: false
}
}
}
--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9
Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found
Encryption Libraries
- sodium: 3.0.2
- libsodium-wrappers: not found
- tweetnacl: not found
FFmpeg
- version: 4.2.4-1ubuntu0.1
- libopus: yes
--------------------------------------------------
Relevant client options:
It isn't really clear how a VoiceReceiver will behave once a connection is disconnected/destroyed. This should be thought about and cleared up. As of now, nothing happens and streams are not destroyed in this instance.
Instead of listening for the raw
event from a Discord.js Client, the hook should listen for the VOICE_SERVER_UPDATE
and VOICE_STATE_UPDATE
event forwarded by client.ws
as this reduces overhead.
To stay consistent with the rest of the discord.js organisation, American English should be used instead of British English.
For example, audio resources follow behaviour
as opposed to behavior
.
The entersState
helper should be exported from the main library since many people will find it useful. It makes a good replacement for Events.once
with AbortControllers for those who aren't on Node.js v15.
If a VoiceConnection
is destroyed (i.e. cannot be reused), then it should be unsubscribed from any audio player it is connected to. This will prevent a memory leak where a destroyed stream will still be kept by an audio player unless manually removed.
The repo should be updated to have documentation that is compatible with the documentation generator, and then the site repository https://github.com/discordjs/site should also be updated to use this documentation.
Currently, you can only pass a string, name
, as metadata that can be attached to a track. This should be replaced with metadata
, that allows a user to pass whatever they want in this place.
As spotted by @almostSouji, discord/discord-api-docs#1741 shows that the endpoint in a VOICE_SERVER_UPDATE packet can sometimes be null.
Continuing on from discordjs/discord.js#3305, the transcoding performance can be improved. Currently, for arbitrary inputs with no inline volume transformer, the following pipeline is used:
Input -> FFmpeg transcoder -> S16LE PCM -> Opus Encoder -> Opus Output
This proposed change would give the following pipeline which is less computationally expensive (as discussed in the PR, transcode time was reduced by around 10%):
Input -> FFmpeg transcoder -> Ogg Opus -> Ogg Opus Demuxer -> Opus Output
This pathway should be avoided for streams that use inline volume, as it would be more computationally expensive.
Please describe the problem you are having in as much detail as possible:
After joining a voice channel, you are unable to switch the voice channel programmatically without severing the connection and starting over.
Include a reproducible code sample here, if possible:
const connection = joinVoiceChannel(channel1);
// wait some time
joinVoiceChannel(channel2);
// nothing happens
When playing the audio from dropbox, the playback ends unexpectedly in the middle of the audio without throwing an error.
However, when playing the downloaded audio file, it will play to the end without any issue.
ends unexpectedly
const connection = joinVoiceChannel({
guildId: guild.id,
channelId: vc.id,
adapterCreator: guild.voiceAdapterCreator,
selfMute: false,
debug: true,
});
connection.on("debug",console.log);
connection.on("error",console.error);
const resource = createAudioResource("https://www.dropbox.com/s/5f1l6seztxvq373/Cartoon%20-%20On%20_%20On%20%28feat.%20Daniel%20Levi%29%20_NCS%20Release_.mp3?dl=1", {
inputType: StreamType.Arbitrary,
});
const player = createAudioPlayer({
behaviors: {
noSubscriber: NoSubscriberBehavior.Pause,
},
});
player.on("debug",console.log);
player.on("error",console.error);
player.play(resource);
const promises = [];
promises.push(entersState(connection, VoiceConnectionStatus.Ready, 1000 * 10));
promises.push(entersState(player, AudioPlayerStatus.AutoPaused, 1000 * 10));
await Promise.all(promises);
connection.subscribe(player);
await entersState(player, AudioPlayerStatus.Playing, 100);
await entersState(player, AudioPlayerStatus.Idle, 2**31-1);
connection.destroy();
play to the end
const resource = createAudioResource(fs.createReadStream("./audio.mp3"), {
inputType: StreamType.Arbitrary,
});
Windows 10
--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9
Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found
Encryption Libraries
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: 1.0.3
FFmpeg
- version: 4.3.1-2020-11-19-essentials_build-www.gyan.dev
- libopus: yes
--------------------------------------------------
Ubuntu 20.04(WSL2)
--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9
Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found
Encryption Libraries
- sodium: not found
- libsodium-wrappers: not found
- tweetnacl: 1.0.3
FFmpeg
- version: 4.2.4-1ubuntu0.1
- libopus: yes
--------------------------------------------------
Relevant client options:
play to the end
state change:
from {"status":"idle","resource":false,"stepTimeout":false}
to {"status":"buffering","resource":true,"stepTimeout":false}
state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
to {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
[NW] [WS] >> {"op":0,"d":{"server_id":"750031320205230311","user_id":"735835853372522546","session_id":"6a2f514b36ee6e18b5693d2695231a30","token":"64e47b95c802a107"}}
[NW] state change:
from {"code":0,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
to {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
[NW] [WS] << {"op":8,"d":{"v":4,"heartbeat_interval":13750.0}}
[NW] [WS] << {"op":2,"d":{"streams":[{"type":"video","ssrc":56152,"rtx_ssrc":56153,"rid":"","quality":0,"active":false}],"ssrc":56151,"port":50008,"modes":["aead_aes256_gcm_rtpsize","aead_aes256_gcm","xsalsa20_poly1305_lite_rtpsize","xsalsa20_poly1305_lite","xsalsa20_poly1305_suffix","xsalsa20_poly1305"],"ip":"103.194.167.85","experiments":["bwe_conservative_link_estimate","bwe_remote_locus_client"]}}
[NW] state change:
from {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":false}
to {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
[NW] [WS] >> {"op":1,"d":{"protocol":"udp","data":{"address":"124.85.238.198","port":52993,"mode":"xsalsa20_poly1305_lite"}}}
[NW] state change:
from {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
to {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
[NW] [WS] << {"op":4,"d":{"video_codec":"H264","secret_key":[33,61,144,252,209,45,62,11,64,2,42,73,159,84,147,237,54,111,233,159,57,197,206,113,90,61,226,185,112,190,236,253],"mode":"xsalsa20_poly1305_lite","media_session_id":"38ee242ae2f8fe471d9f063b187675bd","audio_codec":"opus"}}
[NW] state change:
from {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151}}
to {"code":4,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"64e47b95c802a107","sessionID":"6a2f514b36ee6e18b5693d2695231a30","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56151,"encryptionMode":"xsalsa20_poly1305_lite","secretKey":{"0":33,"1":61,"2":144,"3":252,"4":209,"5":45,"6":62,"7":11,"8":64,"9":2,"10":42,"11":73,"12":159,"13":84,"14":147,"15":237,"16":54,"17":111,"18":233,"19":159,"20":57,"21":197,"22":206,"23":113,"24":90,"25":61,"26":226,"27":185,"28":112,"29":190,"30":236,"31":253},"sequence":48630,"timestamp":2216838939,"nonce":0,"nonceBuffer":{"type":"Buffer","data":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]},"speaking":false,"packetsPlayed":0}}
state change:
from {"status":"autopaused","missedFrames":0,"playbackDuration":100,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":100,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
[NW] [WS] >> {"op":5,"d":{"speaking":1,"delay":0,"ssrc":56151}}
[NW] [WS] >> {"op":3,"d":1623501476079}
[NW] [WS] << {"op":6,"d":1623501476079}
[NW] [WS] >> {"op":3,"d":1623501489832}
[NW] [WS] << {"op":6,"d":1623501489832}
[NW] [WS] >> {"op":3,"d":1623501503589}
[NW] [WS] << {"op":6,"d":1623501503589}
[NW] [WS] >> {"op":3,"d":1623501517342}
[NW] [WS] << {"op":6,"d":1623501517342}
[NW] [WS] >> {"op":3,"d":1623501531100}
[NW] [WS] << {"op":6,"d":1623501531100}
[NW] [WS] >> {"op":3,"d":1623501544858}
[NW] [WS] << {"op":6,"d":1623501544858}
[NW] [WS] >> {"op":3,"d":1623501558620}
[NW] [WS] << {"op":6,"d":1623501558620}
[NW] [WS] >> {"op":3,"d":1623501572377}
[NW] [WS] << {"op":6,"d":1623501572377}
[NW] [WS] >> {"op":3,"d":1623501586135}
[NW] [WS] << {"op":6,"d":1623501586135}
[NW] [WS] >> {"op":3,"d":1623501599891}
[NW] [WS] << {"op":6,"d":1623501599891}
[NW] [WS] >> {"op":3,"d":1623501613646}
[NW] [WS] << {"op":6,"d":1623501613646}
[NW] [WS] >> {"op":3,"d":1623501627399}
[NW] [WS] << {"op":6,"d":1623501627399}
[NW] [WS] >> {"op":3,"d":1623501641152}
[NW] [WS] << {"op":6,"d":1623501641152}
[NW] [WS] >> {"op":3,"d":1623501654912}
[NW] [WS] << {"op":6,"d":1623501654912}
[NW] [WS] >> {"op":3,"d":1623501668669}
[NW] [WS] << {"op":6,"d":1623501668669}
[NW] [WS] >> {"op":5,"d":{"speaking":0,"delay":0,"ssrc":56151}}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":208120,"resource":true,"silencePacketsRemaining":0,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}
ends unexpectedly
state change:
from {"status":"idle","resource":false,"stepTimeout":false}
to {"status":"buffering","resource":true,"stepTimeout":false}
[NW] [WS] >> {"op":0,"d":{"server_id":"750031320205230311","user_id":"735835853372522546","session_id":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","token":"8033b12c62e8ac02"}}
[NW] state change:
from {"code":0,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
to {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
[NW] [WS] << {"op":8,"d":{"v":4,"heartbeat_interval":13750.0}}
[NW] [WS] << {"op":2,"d":{"streams":[{"type":"video","ssrc":56325,"rtx_ssrc":56326,"rid":"","quality":0,"active":false}],"ssrc":56324,"port":50008,"modes":["aead_aes256_gcm_rtpsize","aead_aes256_gcm","xsalsa20_poly1305_lite_rtpsize","xsalsa20_poly1305_lite","xsalsa20_poly1305_suffix","xsalsa20_poly1305"],"ip":"103.194.167.85","experiments":["bwe_conservative_link_estimate","bwe_remote_locus_client"]}}
[NW] state change:
from {"code":1,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":false}
to {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
[NW] [WS] >> {"op":1,"d":{"protocol":"udp","data":{"address":"124.85.238.198","port":50595,"mode":"xsalsa20_poly1305_lite"}}}
[NW] state change:
from {"code":2,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
to {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
[NW] [WS] << {"op":4,"d":{"video_codec":"H264","secret_key":[79,99,160,224,227,8,92,61,7,169,69,57,136,222,196,243,197,72,173,125,243,208,3,252,35,68,147,254,167,60,28,236],"mode":"xsalsa20_poly1305_lite","media_session_id":"38ee242ae2f8fe471d9f063b187675bd","audio_codec":"opus"}}
[NW] state change:
from {"code":3,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324}}
to {"code":4,"ws":true,"connectionOptions":{"endpoint":"japan996.discord.media:443","serverID":"750031320205230311","token":"8033b12c62e8ac02","sessionID":"232eb0ae60b9e7bdbc536d4dcfdb1d4e","userID":"735835853372522546"},"udp":true,"connectionData":{"ssrc":56324,"encryptionMode":"xsalsa20_poly1305_lite","secretKey":{"0":79,"1":99,"2":160,"3":224,"4":227,"5":8,"6":92,"7":61,"8":7,"9":169,"10":69,"11":57,"12":136,"13":222,"14":196,"15":243,"16":197,"17":72,"18":173,"19":125,"20":243,"21":208,"22":3,"23":252,"24":35,"25":68,"26":147,"27":254,"28":167,"29":60,"30":28,"31":236},"sequence":21329,"timestamp":2610706401,"nonce":0,"nonceBuffer":{"type":"Buffer","data":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]},"speaking":false,"packetsPlayed":0}}
state change:
from {"status":"buffering","resource":true,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"stepTimeout":false}
to {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
state change:
from {"status":"autopaused","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
to {"status":"playing","missedFrames":0,"playbackDuration":0,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
[NW] [WS] >> {"op":5,"d":{"speaking":1,"delay":0,"ssrc":56324}}
[NW] [WS] >> {"op":3,"d":1623502100140}
[NW] [WS] << {"op":6,"d":1623502100140}
[NW] [WS] >> {"op":3,"d":1623502113901}
[NW] [WS] << {"op":6,"d":1623502113901}
[NW] [WS] >> {"op":3,"d":1623502127659}
[NW] [WS] << {"op":6,"d":1623502127659}
[NW] [WS] >> {"op":5,"d":{"speaking":0,"delay":0,"ssrc":56324}}
state change:
from {"status":"playing","missedFrames":0,"playbackDuration":48520,"resource":true,"silencePacketsRemaining":5,"stepTimeout":false}
to {"status":"idle","resource":false,"stepTimeout":false}
When running npm pack --dry
to see what files will be included in the final tar, it includes various useless assets such as the .github
folder, examples
folder, tsconfig.json
, etc.
This can be solved by declaring a files
array in the package.json to only include necessary files.
If you attempt to play a resource that has ended or been destroyed, the audio player will transition to the Playing state, but then immediately to the Idle state since the resource can't actually be played.
It would be better if it just stayed in the state it's currently in if the resource isn't playable, and instead an error should be thrown.
Currently, many of the classes are imported from their files, which isn't ideal.
Because peer dependencies are automatically installed in npm v7, it doesn't make sense to list all the audio dependencies in there as they will all be installed when only one from each category is needed at most.
Instead, a notice should be put in the README/documentation that states the supported versions of the dependencies, and informs the user that they need to manually install the dependencies.
Please add documentation or an example for Javascript.
So that the documentation can be parsed correctly by https://github.com/discordjs/docgen/, the documentation should be converted from TSDoc to JSDoc.
A fairly simple starter example should be created to show users the basics of using this library:
The library should implement existing voice receive logic from discord.js even though the this is officially undocumented.
selfDeaf
to false when joining a voice channel (to receive packets)Currently, @discordjs/voice
hooks directly into Discord.js. It would be nicer if the users could set their own adapters so they could use it with other Discord API libraries or some other arbitrary gateway connection.
This documentation will serve to:
The library takes the following approach to disconnects (i.e. the voice websocket closing):
If the close code is 4015 or a non-Discord close code (anything <4000), then a new websocket is opened and it will try to resume the session (see voice close code documentation)
If this fails, then the websocket will disconnect once again, and lead to the scenario below.
If the close code is 4014, the voice connection will enter a disconnected state. At the moment I am finding this problematic because of discord/discord-api-docs#2450.
A bot being moved into a different voice channel on the same guild can trigger a close with the code 4014 and it will reconnect automatically as new voice server and state packets are sent. However, the same close code is emitted for a bot being kicked from the guild or disconnected from the voice channel.
The issue here is that if the bot is moved to a voice channel, the disconnected event will be emitted only for the client to automatically reconnect itself a few moments later, whereas the latter two cases will stay disconnected unless the user intervenes.
This isn't really friendly for the user when they're trying to figure out whether their bot has actually disconnected from a guild's voice channels, or if they just moved channels.
The final case is for Discord codes that are not 4014 or 4015. For these codes, the bot will automatically try to rejoin the channel and will enter the Signalling state.
There should be some effort in documenting this to make it as easy as possible for the user to handle disconnects properly. As well as documenting this, example code that addresses the above issues would be helpful too.
It would be nice to have the following utilities:
AudioReceiveStream
to help make sense of the data (e.g. to aid with jitter buffering or packet reordering)It would be nice if discord can automatically end the voice call if there is only participant in the group call(which had 2 or more participants obviously) for more than 2 minutes.
Thank you so much
Add an information in the player about time streamed.
such as <Player>.streamTime
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.