giorgosera/pythia

pythia's Issues

What's next with visualizations

First of all after implementing LexRank we would be able to show either the top tweets or a text summary of what happened. Put this summary either on the left hand side of the timeline or right below.

Then we have to create the main timeline with the most important events. Coding this will be challenging but you are awesome so don't worry! And you have D3.js by your side.

Then prettify the timeline (shades and stuff)

Next session

--> I added a threshld value in online clustering. Find out what the best value is
--> Add tests for online clustering. Suspicion of documents getting lost.
-->

Add twitter oauth keys in a config file. They must not be hardcoded

Problem with jpeg support in PIL

How to fix:

1. Uninstall PIL
2. sudo apt-get install libjpeg-dev libpng-dev zlib1g-dev liblcms1-dev python-dev
3. $ cd jpeg-8c
   $ ./configure --enable-shared
   $ make
   $ make instal
4. lnstall PIL again

More info:
http://ygamretuta.me/2011/05/27/install-pil-in-ubuntu-natty-python27-virtualen/

EDIT:
A MUCH BETTER APPROACH IS
1/ Call 'pip install -I pil --no-install' to download and unpack the PIL source into your build directory;
2/ Get into your build directory and edit setup.py;
3/ Find the line that says 'add_directory(library_dirs, "/usr/lib")' (line 214 here);
4/ Add the line 'add_directory(library_dirs, "/usr/lib/i386-linux-gnu")' afterwards;
5/ Call 'pip install -I pil --no-download' to finish the installation.

Index should have methods to remove terms NOT DOCS

Allow text analyser to ignore arabic

Improve the algorithm for sentiment.

Currently we use Alchemy. Too slow and RUBBISH. Perhaps use ur own trained classifiers with nltk

text analyser should return only a feature vector and the rest of the methods must be refactored in a Cluster class

Consider adding porter stemming to improve clustering

Implement LexRank

Document experiments

Visualize topic growth

Implement centroid based summarizer

Implement breadth first search to find all followers

See book example

Make urls appear in term vector

Implement evaluation framework

Try to create a feature vector to be used only with Jaccard

Implement user classification