brainicism/discorddataparser

Utility to parse and visualize raw Discord data request information

what do i put here KPOP ANIME FORTNITE

discorddataparser's Issues

Copy .css file to output directory rather than using relative path

undefined local variable or method `__dir__' for #<ResultRenderer:0x0000000160f558>

Begin parsing activity...
Progress: 1/3 (events-2016-00000-of-00001.json)
Progress: 2/3 (events-2017-00000-of-00001.json)
Progress: 3/3 (events-2018-00000-of-00001.json)
Finished parsing activity...
undefined local variable or method `__dir__' for #<ResultRenderer:0x0000000160f558>

Running this on Window's Linux Subsystem.

 ruby app.rb --data-path="./discord-data-2018-08-06/"

There's only index.css in output/visualizations/ Other output folders seem populated.

Check the ruby version at the start and throw an error if it's < 2.0

Could do this instead of #42

Investigate ????? reactions

Failed while parsing messages

Progress: 303/484 (Direct Message with bxllona / tchai#6555)
No such file or directory - ./output/prettified/messages/Direct Message with bxllona / tchai#6555_387451607731666944.txt.txt

This particular issue seemed to pass when I removed the entry from index.json, and @taahamahdi got it to run it after removing the entry.

Look into builds for linux/mac

Switch to github releases

Display the timezone being used in the visualization

ERB template not decoding emoji characters correctly

Add flag to rebuild binary via ocra

Add favicon.ico for the visualization

Filter commonly used words based on min word length in UI

Currently only set as a command line argument

Output folder isn't created if it doesn't exist

No such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csvNo such file or directory @ rb_sysopen - ./output/messages/by_date.csv

Fill in missing dates for date analyses

Parser currently only outputs data for dates with messages sent, and nothing on days with no messages sent. Need to populate the dates with 0s.

The files in DiscordDataParser\output\prettified\messages are in .txt.txt format

Using Window's Linux Subsystem to run the ruby file. Same thing happens with running it directly and using the app.exe

@taahamahdi ran my data and also had it do the same thing, even thought it doesn't normally, so it's probably my data.

Refactor null data point filling methods

fill_messages_by_date, fill_messages_by_time_of_day , and fill_messages_by_day_of_week inside message_by_date_processor.rb. Each method sorts the array twice while trying to insert 0 values for dates without data points. This can be alternatively be done in-place. Each method serves a very similar purpose, and may be able to be combined into one.

Add "commonly used phrases"

Instead of only looking at individual words, find occurances of combination of words (of varying length) in messages.

i.e.
"Damn I love anime."
"I love anime."
"Do you know I love anime?"

"I love anime" would occur 3 times.

Have adjustable time range in the UI of the visualization

A more QOL suggestion

Restrict data points for certain graphs

Some graphs don't need every single data point provided, i.e commonly used words.

Put user's name in the title

Stored in user.json

Move css and js to separate files

Add command line argument for timezone

Also display current timezone in the UI

Analyze "Message by Time of Day" but by the time at the local location and timezone it was sent from.

For anyone that feel like messing with timezones and enjoy tearing their hair out.

This should be a seperate analysis from the current "Message by Time of Day".

As an example, let's say I send 5 messages.
"I love you" sent from 16:00 UTC-5 (21:00 UTC)
"Just kidding" sent from16:00 UTC-3 (19:00 UTC)
"Heyyyy" sent from 16:00 UTC-8 (00:00 UTC)
"Just chillin wbu" sent from 19:00 UTC-5 (00:00 UTC)
"I have a bf" sent from 19:00 UTC-2 (21:00 UTC)

The "Message by Time of Day" would count 2 of (21:00 UTC), 2 of (00:00 UTC), and 1 of (19:00 UTC).

What I am suggesting is for something to count 3 of (16:00 local) and 2 of (19:00 local).
This could be done by looking at the location that the message was sent from, then figuring out the local timezone from that.

Bonus points for looking at the date of the message and figuring out Daylight Savings and the many quirks of timezones.

This is relevant for co-op students and people that travel often.

Not being able to recognize data-path?

Am I missing something? I cannot analyze the contents of the package after downloading and unzipping the email from Discord, it throws a directory not available when I can definitely cd into it.

Update the "__div__" error message to tell the user to use Ruby > 2.0

#32
Currently the error message is not very informative, and people can't read.

Error handling if ocra isn't installed

Commonly Used Words vs Commonly Used Phrases

Currently implementation of CUW is CUP. Need to distinguish between them or add both of them

Improve title of visualization to not just say [Username]'s Results

Instead of having [Username]'s Results, could say [Username]#XXXX's Results with the image of the user's profile picture on the side.

Output is in ascii (ANSI?) instead of UTF-8

This also happens when I open the prettyified message files in notepad++, until I choose "encode as UTF-8.

I'm not sure if there's meta data in these files to tell you what encoding it is, this might be related to #14

Use loops to create charts html

Stats for specific thread

Put specified thread name (if specified) into message analysis heading

[#19] needs to be done first

Clean up discord custom emotes from message analysis

Currently shows up as <:emotename:emoteid>. Might want to only show :emotename:

'invalid byte sequence in Windows-31J' when using japanese locale

Incorrect Output?

Hi, So i just checked this output for my most active threads, and it looks like this one group chat is the most active, but I wanted to make sure that this was right and to my shock and surprise (gasp), I found that I had actually sent about 200,000 messages in this one channel, but alas its output was nowhere to be seen.

Here is a screenshot of a csv that Discord sent me.

Is there like a cap on the amount of messages that can be parsed or something? Because it seems like this epic thing is being ignored.

Pick appropriate chart types for each type of data

i.e. Time spent by device would be better suited for a pie chart

Explore apexcharts

https://apexcharts.com/features/
Native support for data point partitioning, more chart types

Separate message and activity analysis into different sections

Add remaining information to index.html

Information still being printed out to console