abrowne2/streamfeel

Twitch chat analytics; filter garbage, see what people are feeling, and more using natural language processing.

streamfeel's Issues

<deleted message> appears at the top (stream replay)

When a message becomes deleted w/ <deleted message>; it appears to the top of the chat frame. It still gets deleted after 3 minutes, but still is problematic to the user.

DataViz Color based on Dark Mode

If twitch is in darkmode, the data analytics pane should show the dashboard as white or black, respectively (change font colors as well).

Improve sentiment dataset

[Ongoing] The sentiment dataset is bare and needs improvement.

Message Spam Detection

Similar to r9k, keep track of each message by something like this:
std::chrono::time_point<std::chrono::steady_clock> now();

If the differences between messages <30 seconds, we can compare the messages. If they're identical, the most recent one is spam.

Standardize classification data

The ultimate goal of this issue is to accomplish #3 . After testing, it seems that the bottleneck in transferring persistent categorizer data is in the native messaging system. By separating the large datasets down into quarter-chunks (or more) it should do the trick.

• Serialize utility revamp
  • For segmenting data: iterate over the contents and separate them into (quarter/eighths).
• Abstraction needed to join all the separate chunks together and to decode them.

Previous Chat History

As the slider is moved, the user could be able to see the chat rendered as it appeared at the time.

Improve categorizer construction speed

It takes ~10 sec to read in the data for the relevance and sentiment categorizers and deserialize them. Currently, we're splitting the sentiment data into two chunks to avoid overflow.

Use of deleted function.

I'm not sure why I get this error, none of the references found googling helps the problem. I tried including the cpp that define the class text_categorizer. Same error different line number and function. It was BuildCategorizer, now it is serialize.
Output:
/data/user/0/com.n0n3m4.droidc/files/temp.c: In function 'std::__ndk1::vector serialize(DataInst&)':
/data/user/0/com.n0n3m4.droidc/files/temp.c:105:25: error: use of deleted function 'mitie::text_categorizer& mitie::text_categorizer::operator=(mitie::text_categorizer&&)'
compilation terminated due to -Wfatal-errors.

/// text_categorizer.h was modified to include "encode / decode" to use the serialization/deserialization/
^^ I feel this could be why

Data Analytics; feeling, emotes, and commands

I've setup the emotes so they can be viewed in the chart. I just have to make an interface to choose between the three options; it should be aligned with the time selector div.

Introduce mentions into response format.

We can now find messages that were directed to the user, but we need to introduce those into the format for the content script to parse them.

TODO:

• Check the current format for new messages.
• Verify the case in-sensitive search works as intended.
• Send "current user" with format to native client in all responses.
  • This will involve modifying the existing message format.

Emote canvas pattern rendering

Occasionally, an emote's img representation will not fully load. This is likely due to the pattern being created before it's loaded the chart update trying to reference it.

Chart.js legend with emotes

Intention: Display the emotes based on frequency, in their image form.

Methods:

• In the form of a CanvasPattern, the background "color" of the respective emote is a repeated pattern that contains the emote in image form.
  • This needs each emote to be preloaded and then rendered properly.
• Custom legend next to chart that matches the color to the emote*
• Potentially an "emote splash", where as emojis are received, they splash up in the user's view, like Facebook Live's animation.

*Requires toggling the legend off for sentiment and commands, but creating a legend via generateLegend() and stylizing it.

This accomplishes part of #2 .

Twitch clips not loading properly

Clips are not loading when posted in chat. Possible reasons:

• Chat messages are cloned to prevent auto-removal from the DOM. The original message is rendered to have the clip appear, but cloning loses this format.
• Twitch is broken

Possible solution:

• The original message (might) have the link to the clip. Parse it before it's rendered to clip format.

Special messages appear at top

All special messages have special-message within their class name. Clone them, remove original, and append at the bottom of the chat.

Fix message popping

If the chat is moving very fast, messages that are deemed relevant will be popped off the DOM automatically by Ember.JS (normal twitch). This can be fixed by adding a "copy" of the message to the DOM, only removing it after an arbitrary time interval.

YouTube Live Compatibility

Make StreamFeel compatible with YouTube Live.

Emoji Parsing (Emote analytics)

Whether it is a BetterTwitchTV emoji, or a standard twitch one, we need a way to parse all of the emoji's and get their respective images. It should work like this:

• Analyze message, check for any emoji's
  • Should be enclosed in tags, and if it's BTTV; alt="Text Representation", else: enclosed in plain text.
  • "traverse" the message, either by parsing <> enclosures or by some other method (?)
• Extract each emoji, removing the image tag and respective space with the alt or plain text.
• Compute all emote frequencies and store them accordingly.

Read User Mentions

Allow the user to view their mentions in a concise format. (maybe have notifications)

Message frequency detection

Based on a minute or 30 seconds time interval, messages are scrutinized based on existing messages received within that time period. If they match a percentage similarity, they're not relevant. "Relevance" would otherwise be determined by the text categorizer, but in this case it could be useful.

Toggling chat filter stalls analytics

See title. I've made a fix but need to re-update the datasets as I've also modified the # of chunks they're split into to confirm if it works.

Native message "spam" frequency guard

Imagine there are spammers saying something like:

Watch our better stream at twitch tv/----. *** your fake views and fake subs. We are better than you. Give us free money and screw your dumbass stream go watch ---- on twitch.]

over and over, with very little time in between. If the chat is not in r9k mode or actively moderated, this is some spam.

Consider;

• Keep track of the time in the native module
• Compute similarity of each message within the time interval, if it is greater than a certain threshold (say 90%), it's probably spam.

Incorrectly incrementing index

Line 308-309 of stream_feel.cc -
while(index++ < pos) hrs += time[index];
should be
while(index < pos) hrs += time[index++];

Port to webassembly

PNaCl is being deprecated; port to newest standard "web assembly"

Determine logged in twitch user

To use some of our additional features, the name of the logged in twitch user is needed. This can be done by searching the DOM.

Spam detection not 100%

When messages from 60 seconds before show, they aren't filtered with the unordered_map solution
Also consider:

• Levenshtein Distance

Data viz dashboard improvement

The current menu for viewing data; sentiment, commands, and emoji's needs:

• Timeline to traverse data "footprints". Imagine a line with a 'slider' on it. As you slide, you can choose to view the sentiment, emoji, or command analytics at that point.

• When no data is loaded, we need to show that there is no data. Currently it's showing undefined.

• Show the number of messages which makes up our data.

• An intuitive interface to choose between viewing what people feel/think, commands used, and emoji's.