jakob-l-m/word_frequency_visualization

Visualization of word usage changes over time. Example usage on german covid tweets.

word_frequency_visualization's Issues

Fix bugs

• Error: attribute d: Expected arc flag ('0' or '1') - No apparent problem but needs to be fixed
• coloring after date range change
• dotted lines should not be over the arcs
• prevent empty date range selection
• make stopwords not all lowercase
• Files with Ä,Ö,Ü,ß cannot be opened. replace with AE,OE,UE,SS

View for a date

• Create basic view with a list of the words associated with that day
• Generate Word Clouds
• use actual weights
• Make it look nice
• Animations between Clouds

Animations

Animations for lines, dates, circles etc

• Lines
• Dates
• Circles
• Frequency Graph
• word clouds

Plots/Graphs

• Tweet frequency with label for ÖR and Private
• Tweet frequency for each profile -> data validation
• Function flowchart
• script interactions
• Tweets sorted by weekdays
• each profile grouped by month. subplots (4,3)

Github Doku

Create dock for GitHub

• Main Readme with pics and explanation
• Readme in folders, explaining file structure

Ausarbeitung

Einleitung (2 Seiten - 2)

• Motivation
• Ziele
• Aufbau

Grundlagen (9 Seiten - 11)

• Scraping
• Daten Pipeline
• Input von anderen Datensätzen
• Analyse
• Exportieren der Daten zur Visualisierung

Visualisierung (7 Seiten - 18)

• Erstellen der Grafiken
• Animationen
• Click und Hover Events
• Farbgebung

Ergebnisse und Evaluation (6 Seiten - 24)

• Veröffentlichung
• Probleme der automatisierten Auswahl
• Probleme an Tweets/Online Sprache

Zusammenfassung und Ausblick (4 Seiten - 28)

• Möglichkeiten der Erweiterung
• Abschließende Worte
• Echtzeit

Anhang

• Verwendete Bibliotheken mit Anhang

Search Option

• Textarea to type into
• Find an efficient algorithm
• Suggest words that are close
• Make it pretty

Clean Up

Write comments

• front end
• back end

Remove unused code

• front end
• back end

Other tasks:

• transfer notebooks into .py files
• consistent fonts for website
• add about section

View for a word

• Lable x-Axis correctly
• Calculate Word size by word length
• Create Graph
• unpack weights

Preprocessing

• implement tf-idf
• use a keyword file
• update tweet data
• use a timestamp file for the data base
• Filter out Tweet replies
• create an exporter to output json files
• extract uni, bi and tir grams
• Use TreeTagger

Date selection slider

• Add a slider
• Implement functionality
• Make it look nice
• Display current interval