dani7b/rtwup

Realtime Twitter Users' Profiles

rtwup's Issues

add class topology submitter

add documentation to explain how to build and start the system

e.g. I'd like something like this:

1. build with mvn clean package
2. start redis with the command ... (using port X, etc... )
3. start the socket.io server with command ...
4. launch the executable with java -jar .jar
5. you can see the output of the system at this page

fix UrlExpansionTestCase

when fixed, move the logic to ExpanderBolt

reconfigure spout to be able to filter tweets based on different queries

required queries should include following parameters from
Twitter API:

• track
• locations

Useful documentation passage about OR combination of different query parameters:

The track, follow, and locations fields should be considered to be combined with an OR operator. track=foo&follow=1234 returns Tweets matching "foo" OR created by user 1234.

This determines that, if needed, AND query semantics should be dealt with inside the Topology.

testingUrl should not be re-assigned.

in ExpanderBolt, create a new variable when calling connection.getURL().

Active user profiles counter and downloader

This prototype tracks twitter filtered stream and provides a realtime view of active users in a specific time window.

Moreover, includes a dedicated Bolt that writes user profiles to a Profile Repository.

Distinct active users counter

The system tracks the number of unique users during time windows of predefined size.

It's implemented making use of Redis commands.

It follows the same pattern used until now.

Keys in redis can be something like this:

Hour (?)
2013-09-30_13:00CET ...

Day
2013-09-30 1 2 3 4
2013-10-10 1 2 3 4 5 6

Month
2013-09 1 2 3 4
2013-10 1 2 3 4 5 6

remove and ignore eclipse related project files and directories

.settings
.classpath
.project

Implement a simple Bolt to write users to Profiles Repository

using a library developed by Cybion

migrate json serialisation to gson or jackson

all bolts should take configurations from Map conf

Without doing things like this:

this.pool = new JedisPool(new JedisPoolConfig(), "localhost");

In this way, all configurations are set and passed in at topology construction time.