benfortuna/newsagent

An RSS aggregation library

newsagent's Issues

Support pubsubhubbub

Use pubsubhubbub to publish and subscribe to interesting articles and feeds:

http://code.google.com/p/pubsubhubbub-java/

Original issue reported on code.google.com by benfortuna on 26 Jan 2012 at 11:06

Implement feed store

A feed store provides read operations on feeds, sources and entries.

Should support:

 - getFeedSources()
 - getFeedSources(Context)

 - getFeeds()
 - getFeeds(FeedSource)

 - getEntries(Feed)

Original issue reported on code.google.com by benfortuna on 1 Dec 2011 at 11:52

Investigate using Apache Camel for feed consumption

Potentially a more standardised approach to allow for extensions and 
multi-protocol support:

http://camel.apache.org/rss.html

Original issue reported on code.google.com by benfortuna on 16 Feb 2012 at 1:11

Use the public suffix list to identify feed sources

A feed doesn't necessarily provide information on the source (ie. 
games.slashdot.org doesn't specify slashdot.org anywhere).

Use the public suffix list to identify the registered domain:

http://publicsuffix.org/list/

Original issue reported on code.google.com by benfortuna on 2 Dec 2011 at 6:06

Support functionality similar to deprecated Google Feed API

https://developers.google.com/feed/v1/jsondevguide#sending-a-basic-query

Fix relative URLs in article body

Some feeds (e.g. http://www.pcworld.com/index.rss) use relative URLs. 

When parsing check all <a> nodes for valid URLs, and otherwise modify the HTML 
to create absolute URLs using the feed source link.

Original issue reported on code.google.com by benfortuna on 29 Apr 2013 at 5:07

Android app

Implement an Android app.

Primary focus: podcasts (also support other feeds)

Original issue reported on code.google.com by benfortuna on 19 Oct 2012 at 3:17

Use a proxy simulator for testing proxy support

Something like this:

http://sourceforge.net/projects/wpg-proxy/

Original issue reported on code.google.com by benfortuna on 16 Sep 2013 at 6:51

Support parsing media:thumbnail/media:title

Reddit uses additional elements in links to support thumbnail images:

<item>
...
<media:title>A great idea for dog owners</media:title>
<media:thumbnail url="http://f.thumbs.redditmedia.com/LqpFIyiz8pRYHuCS.jpg"/>
</item>

Original issue reported on code.google.com by benfortuna on 25 Mar 2013 at 3:46

Implement path/uid translation from url

Generate a unique path for each feed / entry link. eg:

import java.security.MessageDigest

import org.bouncycastle.util.encoders.Hex


def url2uid = { url ->
    def uid = url.host.split('\\s*\\.\\s*').reverse() as List
    uid.addAll url.path.split('\\s*/\\s*').findAll { !it.empty }
    def digest = MessageDigest.getInstance('md5')
    def checksum = digest.digest url.toString().bytes
    uid << new String(Hex.encode(checksum))
}

URL url = 
['http://games.slashdot.org/story/11/11/25/2217247/valves-gabe-newell-on-piracy-
its-not-a-pricing-problem?utm_source=rss1.0mainlinkanon&utm_medium=feed']

println url2uid(url)

Original issue reported on code.google.com by benfortuna on 29 Nov 2011 at 2:50

Add more html code names to HtmlDecoder

http://ascii.cl/htmlcodes.htm

Names as follows:

quot
amp
lt
gt
nbsp
iexcl
cent
pound
curren
yen
brvbar
sect
uml
copy
ordf
laquo
not
shy
reg
macr
deg
plusmn
sup2
sup3
acute
micro
para
middot
cedil
sup1
ordm
raquo
frac14
frac12
frac34
iquest
Agrave
Aacute
Acirc
Atilde
Auml
Aring
AElig
Ccedil
Egrave
Eacute
Ecirc
Euml
Igrave
Iacute
Icirc
Iuml
ETH
Ntilde
Ograve
Oacute
Ocirc
Otilde
Ouml
times
Oslash
Ugrave
Uacute
Ucirc
Uuml
Yacute
THORN
szlig
agrave
aacute
acirc
atilde
auml
aring
aelig
ccedil
egrave
eacute
ecirc
euml
igrave
iacute
icirc
iuml
eth
ntilde
ograve
oacute
ocirc
otilde
ouml
divide
oslash
ugrave
uacute
ucirc
uuml
yacute
thorn
yuml
euro

Original issue reported on code.google.com by benfortuna on 30 Jan 2012 at 6:16

Implement rest endpoints

1. Provide REST endpoints for subscribe, get, and unsubscribe.

2. Expose the endpoints as OSGi services

Original issue reported on code.google.com by benfortuna on 9 Oct 2013 at 1:15

Provide AWS lambda functions

Feed aggregation and search functions.

Eg.

• FeedAggregate - load all subscribed feeds and store with metadata (eg. Categories, media type, etc) in nosql storage.

• FeedSearch - query aggregated data based on search criteria

• FeedTruncate - remove older feed items from aggregation

• FeedSubscription - add/remove subscriptions to include in aggregation

Implement subscription grouping and sorting

Grouping:

 - Date: Today, Yesterday, Older Items
 - Domain: .org, .edu, .com, etc
 - Frequency: < 10 posts today, this week, this month

Sorting:

 - Date
 - Title
 - Domain

Original issue reported on code.google.com by benfortuna on 9 Feb 2012 at 4:33

Implement reader UI

Three elements in a vertical layout:

1. Control Ribbon

 * Home: search, navigation
 * View: sort, group, filter
 * Action: flag, tag, share

2. Item View

 * Breadcrumb: Subscriptions > {'All Items' | <saved searches> | <tags> }
 * Item Table: ^^- Feeds         ^^- Entries   ^^- Entries        ^^- Feeds

3. Preview

 * Entry HTML viewer

Original issue reported on code.google.com by benfortuna on 7 Feb 2012 at 11:34