holoarchivists/twclazer

Twitcasting Recorder

twclazer's Issues

Timeout on websocket recv()

Timeout value of 2 seconds for ws.recv() here is probably too low.
While it's usually fast enough mid-download, the stream websocket seems to take a bit more time to respond right after establishing a connection. It causes a loop of continuous reconnects that can go for a few minutes before the server manages to respond fast enough from the start. What's worse, the server doesn't seem to expect a possibility of websocket being closed from client side, since a call to ws.close() after timeout takes 20 seconds, which is twice the default value of close_timeout argument of the connect method.

While this loop of reconnects might happen in the middle of a download, most often it's at the start, and once the program manages to get out of the loop, the rest usually goes without a problem. It seems to happen less often on audio-only streams, and more often on motion-heavy ones, such as videogames.

With all that, I suggest increasing the timeout to at least 20-30 seconds, since avoiding possible stuttering with a short timeout is not worth a guaranteed loss of 20 seconds of footage that follows. And in the worst case this repeats multiple times in a row. I've tested timeout values of 30 and 60 seconds, and it seems to fix the problem and otherwise work fine, except causing the program to take longer to stop on clean exit at the end of the stream, which should probably be acceptable.

Generation of authorization headers used to access frontend api | Error when generating token fails

I suggest that this issue is used to keep track of currently ongoing situation with "x-web-authorizekey" header, so that pieces of information are not scattered around multiple closed PR but present in one place.

Twitcasting uses "x-web-sessionid" and "x-web-authorizekey" client headers to confirm that request to frontendapi subdomain endpoints come from legitimate clients. While first is simple a plaintext value, embedded in html on channel's home page, second is generated based on value (salt, used among other variables to calculate hashsum used as part of "x-web-authorizekey" header) produced by executing some Javascript code in PlayerPage2.js. Since the code is minified and to some extent obfuscated, parsing it might be challenging.

First attempt to handle it by hardcoding the salt value was made in #7. Then it became apparent that it gets changed every few weeks along with updates of normal code in PlayerPage2, and code was updated at #12 to use new value. It worth noting that old salt doesn't become invalid immediately after PlayerPage2.js changes, and keeps being accepted for some time along with a new one.

Then #14 added code for automated extraction, using regexp to extract salt-related code from PlayerPage2.js and javascript module to eval() it using nodejs to get salt value. Regular expression it used was too strict and stopped matching after PlayerPage2.js update and it got reverted back to still valid but now outdated hardcoded value in #15, which is likely to also stop working soon.

Documentation on filename formatting

Documentation on filename formatting options differs from actual state of the code in a few aspects.

Two formatting options for output file name described in help and README, namely Hashtag and Pin Message, are not present in FileFormat_Translations. Category Name value is taken from a field that doesn't exist in server response, and is effectively always None. None of them look like something that would be used in a stream recording filename to me, so the easiest fix would probably be removing them from the documentation.

Category ID currently gets movie id as value instead of category id. Since movie id is pretty useful as part of a filename, and some users could be using %%Ci for that, it would make sense to leave this option to be placeholder for movie id and update documentation to reflect that.

Currently, if more than one stream happens in a single day, both of them will have the same filename generated if default formatting options are used, which leads to both of them to be written in a single file. Solution to that might be adding movie id to default filename format the way yt-dlp does.

Both file name and chat formatting descriptions use single % to precede format placeholders, but formatting functions use %% instead, which is reflected in default format strings for both filename and chat format also using %%. Changing that might, again, break existing setups, so help could be updated to use it as well.