Streamline your Twitch EventSub JavaScript integration with TESjs
Home Page: https://www.tesjs.net/
License: MIT License
My name is Mitch and I am a software engineer from North Carolina.
Check out some of my cool projects!
TESjs is a module for Node.js that streamlines usage of Twitch's EventSub system.
ClipNShip is a browser module that allows consumers to modify video to be in different size formats completely client-side.
MtheBot_ is a chatbot for Twitch which aims to make it easy for streamers to have an interactive experience with their viewers.
Is your feature request related to a problem? Please describe.
Now that some events are being deprecated and receiving new versions, TESjs needs to allow for specification of the subscription version when subscribing to an event.
Describe the solution you'd like
Add a third argument to subscribe for the subscription version
Additional context
https://dev.twitch.tv/docs/eventsub/eventsub-subscription-types/#channelfollow
Is your feature request related to a problem? Please describe.
The current method is not quite right. Should be done a little bit differently to be more correct.
Describe the solution you'd like
Change current method to user the validate endpoint, only request new AAT when current not valid.
Additional context
Twitch Doc
Describe the bug
TypeError: this.refreshToken is not a function
To Reproduce
Steps to reproduce the behavior:
identity with id, secret, acessToken & refreshToken, set listener to type websocket.channel.follow & channel.updateExpected behavior
No exception, refresh token get's renewed without issues.
Screenshots
C:\Users\XXXXXXXXX\git\XXXXXXXXX\node_modules\tesjs\lib\auth.js:121
this.refreshToken();
^
TypeError: this.refreshToken is not a function
at Timeout._validateToken [as _onTimeout] (C:\Users\XXXXXXXXX\git\XXXXXXXXX\node_modules\tesjs\lib\auth.js:121:18)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Node.js v19.6.1
Environment:
Additional context
No additional content.
EventSub WebSockets are nearing the end of beta, and with that the URL is changing. Need to update it and drop the beta tag from TESjs
Describe the bug
Variable name inconsistency, finishedCallback in function declaration, callback when calling recursive.
Is your feature request related to a problem? Please describe.
The Helix endpoints for getSubscriptions has optional query params to get only subscriptions with a certain status or a certain type. This should be possible to use by developers through TESjs.
Describe the solution you'd like
A getSubscriptionsByType() function and a getSubscriptionsByStatus() function that you can pass a string for type or status respectively. As for implementation, it should be similar to the current getSubscriptions function (incl. pagination etc).
Additional context
See Twitch Doc for more info.
Is your feature request related to a problem? Please describe.
A lot of people might use TS to develop their applications.
Describe the solution you'd like
TES should support that by providing type definitions for TES functions. This would be provided through DefinitelyTyped. Maintaining these types will probably be a bit of a pain as EventSub is updated, but not a huge deal. Some examples of things that would need to be added:
Is your feature request related to a problem? Please describe.
There are cases where it may be important to perform some kind of cleanup when a subscription gets revoked
Describe the solution you'd like
Fire a notification for a revocation of a subscription.'
I am getting 400 Bad Request errors for the three events user.update, user.authorization.revoke and revokation:
Error subscribing to user.authorization.revoke {
error: 'Bad Request',
status: 400,
message: "unknown validation error: Key: 'SubscriptionCondition.client_id' Error:Field validation for 'client_id' failed on the 'required' tag"
}
Error subscribing to user.update {
error: 'Bad Request',
status: 400,
message: "unknown validation error: Key: 'SubscriptionCondition.user_id' Error:Field validation for 'user_id' failed on the 'required' tag"
}
Error subscribing to revokation {
error: 'Bad Request',
status: 400,
message: 'invalid subscription type and version'
}
To Reproduce
Steps to reproduce the behavior:
Expected behavior
There should not be any errors but success messages for all three event subscriptions.
Currently it'll just get the first page, which could become and issue for those with many event subscriptions
whserver is old, and can be greatly improved from a code-style perspective. Use this issue to track
Describe the bug
Stemming from #24, we do not need to rely on the verify option in bodyParser.json() to have a buffer available for signature verification. This way, we can eliminate common interference with existing parsing middleware by calling that without options and using a custom middleware which converts the JSON body to a buffer instead.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Any request which has or has not been parsed should still go through without issue.
Additional context
Looking at the source code of body-parser, we can see that if the req._body flag has been set, the parsers skip to the next middleware. This means that no matter what, if we call bodyParser.json() in the TESjs route, the next middleware will have a parsed body guaranteed.
Describe the bug
If the title, prompt, or user_input contains the symbol '&', there is no way to redeem and receive the data.
To Reproduce
In the 'user_input' field, if you enter a query string URL like:
example.com?qs=1 -> success
example.com?qs=1&qs=2 -> failed.
Expected behavior
Screenshots
Environment:
Additional context
I have opened the debug mode, and it displays:
BUG:
Thu, 14 Dec 2023 12:55:06 GMT - TESjs - Verifying webhook request
Thu, 14 Dec 2023 12:55:06 GMT - TESjs - Request contains message signature, calculating verification signature
Thu, 14 Dec 2023 12:55:06 GMT - TESjs - Request message signature 7347c8f00563c55a4f68122e0296291ecce8882312f017f62aebd4f65dedd7dd does not match calculated signature 7dd65c659958415a207b21f890bbeb5deb207d7a7186b5f99228e888eb102fde
Success:
Thu, 14 Dec 2023 13:02:30 GMT - TESjs - Verifying webhook request
Thu, 14 Dec 2023 13:02:30 GMT - TESjs - Request contains message signature, calculating verification signature
Thu, 14 Dec 2023 13:02:30 GMT - TESjs - Request message signature match
Thu, 14 Dec 2023 13:02:30 GMT - TESjs - Received notification for type channel.channel_points_custom_reward_redemption.add
Is your feature request related to a problem? Please describe.
Needs better docs, especially for functions I think
Since the documentation of EventSub events seem to present a different sort order than the sort order of actual EventSub responses it is hard to set up events via this library. The problem is not really an issue of this library, but it makes it difficult to develop.
A better handling with attributes/handle arbitrary sort orders. Some kind of automatic attribute name mapping.
Since this is an inconsistency with the EventSub documentation and I have reported this issue, I hope that the current problem with the Custom Points events will be solved. But it would make this library more robust against changes in the sort order of attributes. Just a nice to have!
Describe the bug
TES will still work if provided incorrect config (see #40). This should not happen, and the constructor should throw errors for missing required properties.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
TES should throw an error on construction so the developer knows to use a valid config.
Additional context
Basically, need to prevent issues like #40
Describe the bug
According to the doc, you should not use your Client Secret as the secret for your EventSub subscriptions. This needs to be changed.
Is your feature request related to a problem? Please describe.
Currently TESjs only does signature verification to validate requests. Make a configurable option to check the timestamp header as well.
Describe the solution you'd like
As a precaution against replay attacks, ensure the timestamp is not older than 10 minutes
Additional context
Per Twitch Documentation
When I trigger a mock event using the Twitch CLI tool
twitch-cli.exe event trigger subscribe -t 265345534 -F http://localhost:8080/teswh/event
I get a [401] Request Sent response. It would be great if this application could handle mock events coming from the CLI tool.
[200] Success for all mock events coming from the CLI tool, which are currently:
subscribeunsubscribegiftcheertransactionadd-rewardupdate-rewardremove-rewardadd-redemptionupdate-redemptionIf there is no way to integrate the Twitch CLI tool, tesjs-specific commands to mock events for the CLI would be very helpful.
in the README.md and the docs is says broadcaster_id it should be broadcaster_user_id
currently use request body, make it simpler by using the header
Write your question
Why is it necessary to search for all events and provide them to users when a subscription event occurs?
Additional context
Put anything that adds context to your question. Environment, goal, etc.
Line 149 in fe4a9f5
When an already registered event (409error) occurs, it always searches for all events and returns the event.
If a duplicate subscription event occurs when there are all 100,000 events,
You must proceed up to 100 seeks.
Without internal caching
If multiple duplicate events occur, systemic problems may occur.
Is there any reason for this?
For those who have already registered
stream.online, stream.offline, channel.update
If an event is registered, you must perform a full search at least 3 times.
Write your question
I have set up TESjs as described in the documentation and added ClientID and Client Secret accordingly.
const tes = new TES({
identity: {
id: process.env.TWITCH_CLIENT_ID,
secret: process.env.TWITCH_CLIENT_SECRET
},
listener: {
baseURL: process.env.BASE_URL,
ignoreOldMessages: false
}
});
And it works fine for all endpoints that don't require authentication or a specific scope. But when I try to subscribe to e.g. a subscription (which requires the channel:read:subscriptions scope), I get the error message, that the subscription is missing proper authorization.
{
error: 'Forbidden',
status: 403,
message: 'subscription missing proper authorization'
}
What is the proper way to get access to those endpoints that require authorization or specific scopes?
Thanks in advance.
Is your feature request related to a problem? Please describe.
The current way TES handles auth, it is entirely internal. This could pose an issue with people that are utilizing multiple Twitch libraries, or are using the API or something outside of TES. There should be a way for TES to use the consumer's auth solution.
Describe the solution you'd like
A solution that I found which I think should work well is what twitch-js has done with their onAuthenticationFailure config option. This would allow consumers of TES to supply their own refresh logic if they choose, which will means they don't have to deal with token conflicts.
This may require some additional identity configuration changes, but I'm not sure what it will look like at the moment.
Need to update Express from 4.18.1 to 4.18.2 to get an update to qs to fix a prototype pollution vulnerability
Is your feature request related to a problem? Please describe.
Currently every request to Twitch made be TES validates the access token first. This could cause some rate-limiting issues (maybe, not sure how those are handled) for users of the library that have large loads of users or something that would cause them to need to make those calls. It also makes every operation requiring the API very inefficient, especially those using pagination.
Describe the solution you'd like
An auth service type thing that manages the app access token and refreshing it. This should also mean that should a request be made where an invalid token is used (is possible, and could also happen with some extremely rare timing), that it is handled as well.
The auth service should (for app access tokens, i.e. for webhooks users), should store the current token and refresh token and schedule token refresh. It should also periodically verify the token. Periodic verification should either happen on some kind of timer, or maybe randomly before some requests are made. A good way to maybe do this is have some kind of request manager that would select some requests to verify every so often
Some possible examples of usage:
import AuthService from "authservicepath";const token = AuthService.getToken();fetch("someurl", {method: "GET", headers: { "Authorization": `Bearer ${token}` }})
.then(res => res.json())
.then(data => {
// do things
})
.catch(err => {
if (err.message === "token not valid") {
AuthService.refreshToken();
// somehow make request again
// maybe this would be a good thing for a request manager to deal with??
}
});Is your feature request related to a problem? Please describe.
Currently if a duplicate notification is received, the event will fire again. TESjs should handle these cases (configurable?)
Describe the solution you'd like
Keep track of a list of message IDs received and ignore ones that have been seen already.
Is your feature request related to a problem? Please describe.
Right now, I have mocked data and use sinon to mock requests to the webhooks server with that data.
Describe the solution you'd like
If possible, use twitch-cli (here) to mock API requests made by TESjs as well as mock events. This would give a more robust "real world" setting for the tests to run in, which should improve the reliability of tests and resiliency of TESjs.
This would require some custom stuff to be done for the GitHub Actions currently in use. Luckily, the GHA Linux environments already have homebrew so installing the CLI as part of an action should be simple enough and wont need to be included as part of TESjs.
This may require a slight change in TESjs itself to change the URL to point to for prod/test environments but other than that, should work pretty well I hope.
I see that tests have been worked on to monkey patch the connection to use a locally running websocket server.
Can we have a simple addition to the new TES(config) process where config would allow us to provide a URL of a locally running Twitch CLI mock server for eventsubs?
By default, it comes up as ..
ws://127.0.0.1:8080/ws
but in many cases, we have to change the ports. Allowing us to pass that in directly would be ideal.
Thoughts?
Is your feature request related to a problem? Please describe.
Currently TESjs is written using CommonJS modules. We should make it more modern.
Describe the solution you'd like
Update TESjs to use ESM syntax
Describe the bug
We can't subscribe to raid event
To Reproduce
Steps to reproduce the behavior:
TypeError: Cannot read properties of undefined (reading '0')
at EventManager.queueSubscription (/www/wwwroot/xxxxxxx/node_mjs/lib/events.js:49:42)
at /www/wwwroot/bot.jesuiskristoff.fr/xxxxxx/tesjs/lib/tes.js:180:38
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
Expected behavior
If it is possible fix the issue
Environment:
Add configuration to what types of messages get logged
EX:
Is your feature request related to a problem? Please describe.
This has already been done in the websockets branch, but it would be good to just get out of the way. TES currently uses a custom request function, but that will not be useful for allowing TES to be a browser module.
Describe the solution you'd like
Need to switch to node-fetch so that when TES becomes available as a browser module we can utilize the fetch API as well.
Write your question
Im getting an error in my console.
TypeError [ERR_HTTP_INVALID_HEADER_VALUE]: Invalid value "undefined" for header "Client-ID"
at ClientRequest.setHeader (node:_http_outgoing:576:3)
at new ClientRequest (node:_http_client:256:14)
at Object.request (node:https:353:10)
at C:\Users\skullgaming31\Desktop\SkulledBot\node_modules\tesjs\lib\utils.js:16:31
at new Promise (<anonymous>)
at request (C:\Users\skullgaming31\Desktop\SkulledBot\node_modules\tesjs\lib\utils.js:10:16)
at C:\Users\skullgaming31\Desktop\SkulledBot\node_modules\tesjs\lib\tes.js:144:17
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
code: 'ERR_HTTP_INVALID_HEADER_VALUE'
}
im not sure if its something i havnt set up up but im not using Client-ID anywhere in my code
Additional context
TESjs Current Version: v0.5.1
Is your feature request related to a problem? Please describe.
I think it makes more sense to resolve from a subscribe call when the handshake with Twitch has succeeded, currently is just resolves on a successful request (but the subscription is unverified). This would mean that the subscription is active and no errors occurred when Twitch attempts the challenge.
Describe the solution you'd like
Instead of resolving on successful request to Twitch, resolve after a successful challenge handshake. Also possibly make 401/403 errors log above debug level.
Additional context
When implementing TESjs with an existing Express server, sometimes the configuration of the existing server can cause issues when validating a request from Twitch to TESjs. This would catch these errors to make debugging easier.
Describe the bug
This only affects projects which are integrating with an existing Express server!!
If using an existing Express server, some middlewares (express.json() for example) interfere with the middleware used in TESjs.
To Reproduce
Steps to reproduce the behavior:
express.json() middleware being used for all routesExpected behavior
Event subscription/notifications should come through fine
Actual behavior
The TESjs middleware does not work correctly, making any request from Twitch fail
EventSub WebSockets has hit GA, and with that subscription quotas per WebSocket connection has increased from 100 to 300. Need to update this
Additional context
https://discuss.dev.twitch.tv/t/eventsub-websockets-are-now-out-of-beta-and-generally-available/46527
https://dev.twitch.tv/docs/eventsub/handling-websocket-events/#subscription-limits
Describe the bug
Currently it only gets by type/condition and attempts id gets by using that.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
You should get the subscription information
Additional context
Problem line:
Line 104 in ed06b94
Write your question
Hello, i need help to fix an error when i subscribe to any envent BUT evnets sub works correctly I recieve all the event i subscribed.
my problem is : Cannot read properties of undefined (reading '0')
Additional context
the complete error :
Mon, 12 Sep 2022 18:19:08 GMT - TESjs - Adding notification listener for type channel.subscription.gift
Mon, 12 Sep 2022 18:19:08 GMT - TESjs - Subscribing to topic with type channel.subscription.gift and condition [object Object]
TypeError: Cannot read properties of undefined (reading '0')
at EventManager.queueSubscription (/var/www/vhosts/xxxxxxx.fr/xxxxxxxxxx.fr/node_modules/tesjs/lib/events.js:49:29)
at /var/www/vhosts/jesuiskristoff.fr/bot.jesuiskristoff.fr/node_modules/tesjs/lib/tes.js:189:62
at new Promise ()
at TES.subscribe (/var/www/vhosts/jesuiskristoff.fr/bot.jesuiskristoff.fr/node_modules/tesjs/lib/tes.js:189:16)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
Environment:
Is your feature request related to a problem? Please describe.
Twitch just released their WebSocket delivery proposal recently, meaning that delivery method should be supported in the near future.
Describe the solution you'd like
Add support for delivery over WebSockets in a way that is the same as the current Webhooks delivery method, just with different configuration. EX:
const TES = require('tesjs');
// initialize TESjs
const tes = new TES({
identity: YOUR_CLIENT_ID, // now not an obj as only client id is relevant to ALL cases of ID
listener: {
type: "websocket" // or, alternatively, "webhook"
// baseURL becomes required only for "webhook" listener type
}
});or alternatively (leaning toward this one)
const TES = require('tesjs');
// initialize TESjs
const tes = new TES({
identity: YOUR_CLIENT_ID, // now not an obj as only client id is relevant to ALL cases of ID
// omitting listener object implies WebSocket delivery (makes listener an optional config param)
// or, at least, omitting the baseURL/port/server params
});Is your feature request related to a problem? Please describe.
I think it would be useful for some people to have subscription information provided when an event happens. Some developers may want to use that info conditionally or something idk.
Describe the solution you'd like
Add additional optional argument to event listener callback which will be the subscription object
tes.on('channel.update', (event, subscription) => {
console.log(`Event from subscription ${subscription.id} has happened`);
});
Additional context
Twitch doc
I think the code will be more readable and easier to maintain if it used async/await syntax. This should be relatively straightforward aside from maybe the subscription queue.
Is your feature request related to a problem? Please describe.
As described in #57, it is pretty unclear why a subscription would fail to be made.
When making a request to subscribe without proper authorization, the message looks like so:
{
error: 'Forbidden',
status: 403,
message: 'subscription missing proper authorization'
}
When making a request to subscribe and there is already and existing subscription that matches the type and condition, the message looks like so:
{
error: 'Conflict',
status: 409,
message: 'subscription already exists'
}
TESjs is too optimistic in assuming developers will know to give correct auth scopes to their client, and that subscriptions are cleaned up and so passes the data naively to the subscription queue assuming it was OK.
Describe the solution you'd like
Provide a more clear error on a non-OK response.
Additional context
#57
Make log messages better indicate origin and status
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.