Giter Site home page Giter Site logo

feedjira's Introduction

Feedjira

Build Status Code Climate

Feedjira is a Ruby library designed to parse feeds.

Installation

Add this line to your application's Gemfile:

gem "feedjira"

Parsing

An example of parsing a feed with Feedjira:

xml = HTTParty.get(url).body
feed = Feedjira.parse(xml)
feed.entries.first.title
# => "Announcing version 3.0"

Specifying parser

If you have the XML and just want to provide a parser class for one parse, you can specify that using parse with the parser option:

Feedjira.parse(xml, parser: MyAwesomeParser)

Adding attributes to all feeds types / all entries types

# Add the generator attribute to all feed types
Feedjira::Feed.add_common_feed_element("generator")
xml = HTTParty.get("http://www.pauldix.net/atom.xml").body
Feedjira.parse(xml).generator
# => "TypePad"

Adding attributes to only one class

If you want to add attributes for only one class you simply have to declare them in the class

# Add some GeoRss information
class Feedjira::Parser::RSSEntry
  element "georss:elevation", as: :elevation
end

# Fetch a feed containing GeoRss info and print them
url = "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/significant_week.atom"
xml = HTTParty.get(url).body
Feedjira.parse(xml).entries.each do |entry|
  puts "Elevation: #{entry.elevation}"
end

Configuration

Parsers

Adding a custom parser

You can insert your own parser at the front of the available parser list by:

Feedjira.configure do |config|
  config.parsers.unshift(MyAwesomeParser)
end

Now when you call Feedjira.parse, MyAwesomeParser will be the first one to get a chance to parse the feed.

Explicitly set all available parsers

Feedjira can be configured to use a specific set of parsers and in a specific order:

Feedjira.configure do |config|
  config.parsers = [
    Feedjira::Parser::ITunesRSS,
    MyAwesomeParser,
    Feedjira::Parser::RSS
  ]
end

Stripping whitespace from XML

Feedjira can be configured to strip all whitespace but defaults to lstrip only:

Feedjira.configure do |config|
  config.strip_whitespace = true
end

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/feedjira/feedjira. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

Projects that use Feedjira

Feedjira is used in some awesome projects around the web - from RSS readers to add-ons and everything in between. Here are some of them:

  • Feedbin: Feedbin bills itself as a fast, simple RSS reader that delivers a great reading experience. It's a paid RSS reader that integrates with mobile apps and it even has a fully featured API!

  • Stringer: Stringer is a self-hosted, anti-social RSS reader. It's an open-source project that's easy to deploy to any host, there's even a one-click button to deploy on Heroku.

  • BlogFeeder: BlogFeeder is a paid Shopify App that makes it easy for you to import any external blog into your Shopify store. It helps improve your store's SEO and keeps your blogs in sync, plus a lot more.

  • Feedbunch: Feedbunch is an open source feed reader built to fill the hole left by Google Reader. It aims to support all features of Google Reader and actually improve on others.

  • The Old Reader: The Old Reader advertises as the ultimate social RSS reader. It's free to start and also has a paid premium version. There's an API and it integrates with many different mobile apps.

  • Solve for All: Solve for All combines search engine and feed parsing while protecting your privacy. It's even extendable by the community!

  • Feedi API: Feedi simplifies how you handle RSS, Atom, or JSON feeds. You can add and keep track of your favourite feed data with a simple and clean REST API. All entries are enriched by Machine Learning and Semantic engines.

  • Breaker: The social podcast app

  • Huginn: Huginn is a system for building agents that perform automated tasks for you online.

Note: to get your project on this list, simply send an email with your project's details.

feedjira's People

Contributors

amatriain avatar archiloque avatar bananastalktome avatar benubois avatar bradhe avatar dependabot[bot] avatar everybodykurts avatar ezkl avatar filipepina avatar filiptepper avatar fudanchii avatar gaffneyc avatar hparker avatar iamnader avatar jfiorato avatar jonallured avatar julien51 avatar knu avatar krasnoukhov avatar krzyzak avatar mahemoff avatar meganemura avatar mikeastock avatar mje113 avatar mockdeep avatar mvz avatar nakajima avatar pauldix avatar rickerbh avatar sferik avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

feedjira's Issues

README issue

the on_success handler needs two parameters. It should be

:on_success => lambda {|url, feed| puts feed.title }

instead of:

:on_success => lambda {|feed| puts feed.title }

Otherwise a warning is raised: warning: multiple values for a block parameter (2 for 1)

Parsing custom feed elements is not possible

Just like there is a way to parse custom elements for entries using
Feedzirra::Feed.add_common_feed_entry_element("wfw:commentRss", :as => :comment_rss)

There is no way to do that for the feed object (one that holds entries). It only exposes title, feed_url, url, entries. There is not way to access the other elements.

Feed Entry ID's in ruby 1.9.1

In feeds which do not have an id pulled, feedzirra is supposed to substitute the url. In ruby 1.9.1, the id variable seems to be taking precedence over the id method in feed_entry_utilities.rb, resulting in nil id's for some feeds (feeds which do not have id's in them). Changing the variable name (to entry_id, for example) and using :as => :entry_id in the parsers for entry id elements (in my fork, http://github.com/bananastalktome/feedzirra) appears to fix the issue.

undefined method `httpdate' for "Wed, 24 Mar 2010 23:11:00 EST":String (NoMethodError)

When parsing http://www.engadget.com/rss.xml, the following error occurs:

/home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:264:in `block in add_feed_to_multi': undefined method `httpdate' for "Wed, 24 Mar 2010 23:11:00 EST":String (NoMethodError)
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:262:in `call'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:262:in `new'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:262:in `add_feed_to_multi'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:182:in `block in update'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:181:in `each'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra/feed.rb:181:in `update'
from /home/myHome/my_rails_project/app/models/feed_entry.rb:16:in `block in update_feeds'
from /home/myHome/my_rails_project/app/models/feed_entry.rb:10:in `each'
from /home/myHome/my_rails_project/app/models/feed_entry.rb:10:in `update_feeds'
from /home/myHome/my_rails_project/lib/daemons/feed_fetcher.rb:19:in `<top (required)>'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `load'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/application.rb:176:in `start_load'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/application.rb:257:in `start'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/controller.rb:69:in `run'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons.rb:139:in `block in run'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `call'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons/cmdline.rb:105:in `catch_exceptions'
from /home/myHome/.rvm/gems/ruby-1.9.1-p378/gems/daemons-1.0.10/lib/daemons.rb:138:in `run'
from lib/daemons/feed_fetcher_ctl:14:in `<main>'

I'm using ruby 1.9.1 and Rails 3.0beta

date can't be parsed error received multiple times

I've been screwing around with this error all morning and I can't determine why this used to work and now it doesn't.

I was importing a feed and it used to work fine but now I get DATE CAN'T BE PARSED : date on every entry

I even go into rib and try to manually set

entry.published = DateTime.now

but I get DATE CAN'T BE PARSED: 2011-08-02T10:39:01-04:00

Using Rails 3.0.9 ruby 1.9.2-p290

Any help would be greatly appreciated.

Unable to use in Mac OS X 10.6.2 (Snow Leopard)

Hi,

Been struggling some days (with curl etc). Now I finally removed as much as possible gems and so on, tried to have fresh start. After that installed '$sudo gem install pauldix-feedzirra' which succeed fine.

Now we are here:

Test file:
$ cat feedzirra.rb

require 'rubygems'
require 'feedzirra'

feed = Feedzirra::Feed.fetch_and_parse("http://feeds.feedburner.com/PaulDixExplainsNothing")

Running it:
$ ruby feedzirra.rb

./feedzirra.rb:4: uninitialized constant Feedzirra (NameError)
from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /Library/Ruby/Site/1.8/rubygems/custom_require.rb:31:inrequire'
from feedzirra.rb:2

Environment:
ruby 1.8.7 (2008-08-11 patchlevel 72) [universal-darwin10.0]

$ which curl
/usr/bin/curl

$ curl --version #so factory made one now used, isn't it?

curl 7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4 OpenSSL/0.9.8k zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz

$ gem list --local

*** LOCAL GEMS ***

actionmailer (2.2.2, 1.3.6)
actionpack (2.2.2, 1.13.6)
actionwebservice (1.2.6)
activerecord (2.3.4, 2.2.2, 1.15.6)
activeresource (2.3.4, 2.2.2)
activesupport (2.2.2, 1.4.4)
acts_as_ferret (0.4.3)
builder (2.1.2)
capistrano (2.5.2)
cgi_multipart_eof_fix (2.5.0)
columnize (0.3.1)
crm114 (1.0.3)
daemons (1.0.10)
dnssd (0.6.0)
fastthread (1.0.1)
fcgi (0.8.7)
ferret (0.11.6)
ffi (0.5.4)
gem_plugin (0.2.3)
highline (1.5.0)
hoe (2.4.0)
hpricot (0.6.164)
json_pure (1.2.0)
libxml-ruby (1.1.2)
linecache (0.43)
mdalessio-dryopteris (0.1.2)
mongrel (1.1.5)
narray (0.5.9.7)
needle (1.3.0)
net-scp (1.0.1)
net-sftp (2.0.1, 1.1.1)
net-ssh (2.0.4, 1.1.4)
net-ssh-gateway (1.0.0)
nokogiri (1.4.1)
pauldix-feedzirra (0.0.18)
pauldix-sax-machine (0.0.14)
rack (1.0.1)
rails (2.2.2, 1.2.6)
rake (0.8.7, 0.8.3)
RedCloth (4.1.1)
ruby-debug (0.10.3)
ruby-debug-base (0.10.3)
ruby-debug-ide (0.4.9)
ruby-openid (2.1.2)
ruby-yadis (0.3.4)
rubyforge (2.0.3)
rubygems-update (1.3.5)
rubynode (0.1.5)
sqlite3-ruby (1.2.5, 1.2.4)
stemmer (1.0.1)
taf2-curb (0.5.4.0)
termios (0.9.4)
wirble (0.1.3)
xmpp4r (0.4)
ya2yaml (0.26)

What's wrong, what to try next?

Invalid pointer

Hi,

I'm wondering if anyone has seen this:
[ ~/develop/ignite/site ] master* $ ./script/console
Loading development environment (Rails 2.3.2)

require 'feedzirra'
=> []
feed = Feedzirra::Feed.fetch_and_parse("http://feeds.feedburner.com/PaulDixExplainsNothing")
src/tcmalloc.cc:186] Attempt to free invalid pointer: 0x210140
Abort trap

Google News RSS only returns 10 items?

Hello -

Firstly, thoroughly enjoyed the speed and ease of use of feedzirra so far.

I am noticing that when I call fetch_and_parse(array_of_urls), the results from Google News RSS feeds and Google Blog Search RSS feeds routinely have only 10 items, even though if you navigate the to URL, there are far more.

For example, if you go to:

http://news.google.com/news?pz=1&ned=us&hl=en&q=earthquake&output=RSS

There can bewell, up to a hundred results, shall we say? Now put this through Feedzirra and we get 10.

require 'rubygems'
require 'feedzirra'

@Feed = Feedzirra::Feed.fetch_and_parse("http://news.google.com/news?pz=1&ned=us&hl=en&q=earthquake&output=RSS")

puts @feed.entries.count

Could this be Google limiting what Ruby/Feedzirra can access? If so, is it possible to change the client identification string for accessing these services?

Cheers,

kez.

Get following error when using the console

This is a Ruby 1.9.1 on a Rails 3.0.0.beta app with the following gems bundled:

Gems included by the bundle:

  • abstract (1.0.0)
  • actionmailer (3.0.0.beta)
  • actionpack (3.0.0.beta)
  • activemodel (3.0.0.beta)
  • activerecord (3.0.0.beta)
  • activeresource (3.0.0.beta)
  • activesupport (3.0.0.beta)
  • arel (0.2.1)
  • builder (2.1.2)
  • bundler (0.9.10)
  • curb (0.6.6.0)
  • erubis (2.6.5)
  • feedzirra (0.0.22)
  • i18n (0.3.5)
  • loofah (0.4.6)
  • mail (2.1.3)
  • memcache-client (1.7.8)
  • mime-types (1.16)
  • nokogiri (1.4.1)
  • rack (1.1.0)
  • rack-mount (0.4.7)
  • rack-test (0.5.3)
  • rails (3.0.0.beta)
  • railties (3.0.0.beta)
  • rake (0.8.7)
  • sax-machine (0.0.15)
  • sqlite3-ruby (1.2.5)
  • text-format (1.0.0)
  • text-hyphen (1.0.0)
  • thor (0.13.4)
  • tzinfo (0.3.16)

/Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta/lib/rails.rb:40:in configuration': undefined methodconfig' for nil:NilClass (NoMethodError)
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/loofah-0.4.6/lib/loofah.rb:86:in <top (required)>' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta/lib/active_support/dependencies.rb:167:inrequire'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta/lib/active_support/dependencies.rb:167:in block in require' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta/lib/active_support/dependencies.rb:537:innew_constants_in'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta/lib/active_support/dependencies.rb:167:in require' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.22/lib/feedzirra.rb:6:in<top (required)>'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:41:in require' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:41:inblock (2 levels) in require'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:36:in each' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:36:inblock in require'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:35:in each' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler/runtime.rb:35:inrequire'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.10/lib/bundler.rb:71:in require' from /Users/nick/Code/sandbox/appolo/config/application.rb:6:in<top (required)>'
from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta/lib/rails/commands.rb:33:in require' from /Users/nick/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta/lib/rails/commands.rb:33:in<top (required)>'
from /Users/nick/Code/sandbox/appolo/script/rails:10:in require' from /Users/nick/Code/sandbox/appolo/script/rails:10:in

'

Great library. Thanks. I am willing to help get feedzirra working with Ruby 1.9.1, just let me know what I can do.

Cheers,
Nick

Cant get Feedzirra to work on OSX 10.6

Setting up another dev machine and i cant seem to get Feedzirra to work.

$ script/console
Loading development environment (Rails 2.3.4)
/Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/rails/gem_dependency.rb:119:Warning:
Gem::Dependency#version_requirements is deprecated and will be removed
on or after August 2010. Use #requirement
Failed to load /usr/local/lib/ruby/gems/1.8/gems/curb-0.6.6.0/lib/curb_core.bundle
/usr/local/lib/ruby/gems/1.8/gems/curb-0.6.6.0/lib/curb_core.bundle
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in require' /Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in require'
/Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:521:in
new_constants_in' /Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in require'
/usr/local/lib/ruby/gems/1.8/gems/curb-0.6.6.0/lib/curb.rb:1
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in
gem_original_require' /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:inrequire'
/Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in
require' /Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:521:in new_constants_in'
/Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in
require' /usr/local/lib/ruby/gems/1.8/gems/pauldix-feedzirra-0.0.18/lib/feedzirra.rb:6 /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require'
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in require' /Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in require'
/Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:521:in
new_constants_in' /Users/toh/src/rssvoice/vendor/rails/activesupport/lib/active_support/dependencies.rb:156:in require'
/Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/rails/gem_dependency.rb:208:in
load' /Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:307:in load_gems'
/Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:307:in
each' /Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:307:in load_gems'
/Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:169:in
process' /Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:113:in send'
/Users/toh/src/rssvoice/config/../vendor/rails/railties/lib/initializer.rb:113:in
run' /Users/toh/src/rssvoice/config/environment.rb:13 /usr/local/lib/ruby/1.8/irb/init.rb:252:inrequire'
/usr/local/lib/ruby/1.8/irb/init.rb:252:in load_modules' /usr/local/lib/ruby/1.8/irb/init.rb:250:ineach'
/usr/local/lib/ruby/1.8/irb/init.rb:250:in load_modules' /usr/local/lib/ruby/1.8/irb/init.rb:21:insetup'
/usr/local/lib/ruby/1.8/irb.rb:54:in `start'
/usr/local/bin/irb:13
Missing these required gems:
pauldix-feedzirra

You're running:
ruby 1.8.6.111 at /usr/local/bin/ruby
rubygems 1.3.6 at /Users/toh/.gem/ruby/1.8, /usr/local/lib/ruby/gems/1.8

Run rake gems:install to install the missing gems.

Feedzirra is installed.

Error in Documentation

I was trying to use feedzirra with the on_success callback, but kept getting the following message:
warning: multiple values for a block parameter (2 for 1), and the code would jump to the on_failure block.

Checking the source code for feed.rb, shows that the on_success callback accepts two parameters, url and feed, whereas the documentation only states feed.

An update to the documentation might be in order. Thank you for the great work

Screwy load paths

When building a Rails app, I put my own core extension methods in lib/core_ext and require those files with a core_ext.rb initializer. The problem is that when a lib/core_ext/string.rb file exists in my Rails app, feedzirra loads that one file instead of its own lib/core_ext/string.rb file, which takes away the String#sanitize method.

Duplicate Entries

Hi all -
I have an rss aggregator project where I'm pretty much using this exact
methodology: http://asciicasts.com/episodes/168-feed-parsing.
I have a chron task that runs update_from_feed_continuously, with my Feed.rb
model looking like this:

def self.update_from_feed_continuously()
find(:all).each do |feed|
feed_data = Feedzirra::Feed.fetch_and_parse(feed.feed_url)
feed.add_entries(feed_data.entries)
feed.last_modified = Time.now
feed.save
end
end

def add_entries(entries)
entries.each do |entry|
unless feed_entries.exists? :rss_id => entry.id
entry = feed_entries.build(
:title => entry.try(:title),
:summary => (if entry.summary
Sanitize.clean(entry.summary)
else
""
end),
:url => entry.url,
:published_at => entry.published,
:rss_id => entry.id
)
unless entry.save
logger.debug("::::::feed didn't save:::::")
end
end
end
end

This works PERFECTLY for all but one feed: The Weekly Standard
(http://theweeklystandard.com/rss/site.xml). It basically makes two to three
entries for each item in the xml feed. I want to be able to fix it, or provide
feedback to TWS to tell them why it isn't working. I'm not 100% sure how the
entry.id is being gathered from the feed to be evaluated for the exists? method.
Is it because the feed is missing an actual node? If so, surely we can grab
the instead?
I'm a noob with Rails, so any insight would be very helpful! Thanks!

Timeout settings

Hello
I am using feedzirra on my heroku app. Since I run on a free slice I can not run background jobs and have then made a cronjob from a second machine that updates the rss feeds. The problem there is that each request can not exceed 30secs, or the app will timeout.

I am compensating for this when looping through my feeds, but I have no way to set a timeout on the actual fetch_and_parse command. What is the default timeout for this? Is there a way to set the timeout?

/MartOn

Doesn't work properly with Rails 3?

I get the uninitialised constant error when using it. I've seen other people saying about the issue with loofah causing it. I haven't installed loofah so any ideas what's causing it?

Rails3 active_support/core_ext/time

Rails 3, Ruby 1.9.2

On the require 'feedzirra' I get:

no such file to load -- active_support/core_ext/time

This seems to be related to issue 19?

Here is an updated hack:

--- feedzirra.rb    2010-09-27 02:33:51.000000000 -0500
+++ feedzirra.rb.works  2010-09-27 02:43:42.000000000 -0500
@@ -8,8 +8,9 @@ require 'sax-machine'
 require 'dryopteris'
 require 'uri'
 require 'active_support/basic_object'
-require 'active_support/core_ext/object'
-require 'active_support/core_ext/time' 
+#require 'active_support/core_ext/object'
+require 'active_support/core_ext'
+require 'active_support/time'

 require 'core_ext/date'
 require 'core_ext/string'

An Atom feed is incorrectly marked as an RSS feed

This feed: http://anthonyssong.blogspot.com/feeds/posts/default
Has the string "rdf" in the first 2000 characters which satisfies the able_to_parse? regex in RSS

xml =~ /<rss|rdf/

Thus the RSS parser is chosen as its overlord. However, its definitely an Atom feed. It matches because the Author has "rdf" in their email address.

cirillo@4wardfast....

Is there a reason why there is no < anchoring pattern in front of the "rdf" portion in the regex, e.g. why isnt it:

xml =~ /<rdf|<rdf/

When I make that change to the regex locally it works (that regex fails and ultimately the Atom parser is identified)

Problem implementing iTunes nested categories

Paul,

I started implementing the nested iTunes categories, but when I try to
get the name of the nested category I'm getting the name of the parent
instead.

I have a branch up on github where you can see what I'm doing.

http://bit.ly/3eBz9m

'Feedzirra::Parser::ITunesRSS parsing should parse categories' FAILED
expected: "Gadgets",
got: "Technology" (using ==)
Diff:
@@ -1,2 +1,2 @@
-Gadgets
+Technology

Any suggestions on what I may be doing wrong would be appreciated.

undefined method `sanitize!'

When I include feedzirra v0.0.18 - .23 into a Rails 2.3.4 project, I kept getting errors like it the String.sanitize! method was not found.

I'm guessing core_ext/string.rb conflicts with some other file of the same name elsewhere in my call require chain...Can you rename that include to something less similar to active_support files?

Fixing pauldix-feedzirra project on OS X Snow Leopard

Hello,
My rails project kept saying I was missing feedzirra, but indeed I had it. Looking down the rake output, I saw this error:

dlopen(/Users/misha/.gem/ruby/1.8/gems/taf2-curb-0.5.4.0/lib/curb_core.bundle, 9): no suitable image found. Did find:
/Users/misha/.gem/ruby/1.8/gems/taf2-curb-0.5.4.0/lib/curb_core.bundle: mach-o, but wrong architecture - /Users/misha/.gem/ruby/1.8/gems/taf2-curb-0.5.4.0/lib/curb_core.bundle

Reinstalling taf2-curb with the right architecture flags didn't help. But I think it was because I had taf2-curb installed twice. Uninstalling both, and then installing it again:

sudo env ARCHFLAGS="-arch x86_64" gem install taf2-curb

Fixed the issue. Well for me actually I had to reinstall sqlite3 for some reason:

sudo env ARCHFLAGS="-arch x86_64" gem install sqlite3-ruby

But that was probably a different unrelated issue.

Misha

undefined method `response_code' for #

I am using feedzirra to parse feeds but occassionally see this errors below for many different feeds. It happens randomly. Do you have any idea what the cause could be and potentially can you catch it.

message: "undefined method `response_code' for #"
stacktrace: |-
  /opt/local/lib/ruby/gems/1.8/gems/pauldix-feedzirra-0.0.16/lib/feedzirra/feed.rb:240:in `add_url_to_multi'
      /opt/local/lib/ruby/gems/1.8/gems/pauldix-feedzirra-0.0.16/lib/feedzirra/feed.rb:135:in `call'
      /opt/local/lib/ruby/gems/1.8/gems/pauldix-feedzirra-0.0.16/lib/feedzirra/feed.rb:135:in `perform'
      /opt/local/lib/ruby/gems/1.8/gems/pauldix-feedzirra-0.0.16/lib/feedzirra/feed.rb:135:in `fetch_and_parse'

Segmentation fault

Feedzirra seems to seg fault here quite a bit
/home/tester/test/vendor/gems/gems/feedzirra-0.0.19/lib/feedzirra/feed.rb:205: [BUG] Segmentation fault

We have run into this problem multiple times this week.

Running on ruby 1.8.6 (2007-09-24) [i486-linux]. Any thoughts/suggestions? Thx in advance.

Incompatibility with Rails 3

Using Rails 3 and Ruby 1.9.1 and installing Feedzirra with Bundler. It installs correctly but as soon as I try to execute any Rails command I run into the following error.

$ rails c
/home/abel/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta4/lib/rails.rb:44:in configuration': undefined methodconfig' for nil:NilClass (NoMethodError)
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/loofah-0.4.7/lib/loofah.rb:89:in <top (required)>' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:212:inrequire'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:212:in block in require' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:198:inblock in load_dependency'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:554:in new_constants_in' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:198:inload_dependency'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/activesupport-3.0.0.beta4/lib/active_support/dependencies.rb:212:in require' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/feedzirra-0.0.23/lib/feedzirra.rb:6:in<top (required)>'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:46:in require' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:46:inblock (2 levels) in require'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:41:in each' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:41:inblock in require'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:40:in each' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler/runtime.rb:40:inrequire'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/bundler-0.9.26/lib/bundler.rb:89:in gem_require' from /home/abel/web/feedsmash/config/application.rb:7:in<top (required)>'
from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta4/lib/rails/commands.rb:21:in require' from /home/abel/.rvm/gems/ruby-1.9.1-p378/gems/railties-3.0.0.beta4/lib/rails/commands.rb:21:in<top (required)>'
from script/rails:6:in require' from script/rails:6:in

'

Uninstall Feedzirra and it runs smoothly. This is the same error I get when using other gems that are still incompatible with Rails 3 so I guess that's the problem.

Can feedzirra discover feeds from a blog url?

Would love to see this ability, where a user could type in a blog url OR a feed url. If it's a blog url, feedzirra could discover and use the first feed listed. Many website users don't understand the difference between their RSS feed and their blog url.

bug in update_from_feed

the .any? block short circuits when something returns as true, so only the first attribute of the feed is updated.

    def update_from_feed(feed)
      self.new_entries += find_new_entries_for(feed)
      self.entries.unshift(*self.new_entries)

      updated! if UPDATABLE_ATTRIBUTES.any? do |name|
        update_attribute(feed, name)
      end
    end

youtube atom feed won't parse entries

The feed: http://gdata.youtube.com/feeds/base/standardfeeds/top_favorites?client=ytapi-youtube-index&time=today&v=2 has 0 entries according to feedzirra.

It does validate according to: http://beta.feedvalidator.org/check.cgi?url=http%3A%2F%2Fgdata.youtube.com%2Ffeeds%2Fbase%2Fstandardfeeds%2Ftop_favorites%3Fclient%3Dytapi-youtube-index%26time%3Dtoday%26v%3D2.

It seems to be a combination of a lot of link tags and the atom id tags thats causing this, but I can't pinpoint the exact cause because I am not very familiar with the parser.

I've also posted this in google groups: http://groups.google.com/group/feedzirra/browse_thread/thread/84b4133813819219

taf2-curb

Is there not a way to specify the taf2-curb for requirement in the gemspec somehow?

uninitialized constant Feedzirra

Hi, I get this error when running a simple example such as this one:

require 'rubygems'
require 'feedzirra'
feed = Feedzirra::Feed.fetch_and_parse("http://www.mywebsite.com/rss")

Here is the complete stacktrace:
./feedzirra.rb:6: uninitialized constant Feedzirra (NameError)
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in gem_original_require' from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:inrequire'
from feedzirra.rb:3

Could you please help me ?

Custom Parsing Twitter Atom

If I want to parse a Twitter Atom feed, which returns to_user, from_user, profile_image_url, etc, do I need to write my own parser? Is this necessary, and has it not already been done?

Update: I am trying to write a custom parser, and keep getting uninitialized constant in the console when I try to add my new feed class. I added the appropriate 'require' lines to the beginning of lib/feedzirra.rb. Any help would be appreciated.

Thanks!

namespaces in custom attribute parsing appear broken in Ruby 1.9

Am not exactly sure what's going on, but this works correctly (returns a URI value from the attribute) in 1.8.7, but returns nothing in 1.9.2:

# add custom parsing logic for Google Reader Feeds
Feedzirra::Feed.add_common_feed_entry_element(:id, :value => :"gr:original-id", :as => :original_id)

I am assuming this has something to do with namespace handling.

Trouble installing Feedzirra (Loofah?) on Rails 3

Hi,
I'm trying to use Feedzirra on a Rails 3 project, and I'm having trouble getting it to install correctly. I tried installing Loofah 0.4.7 and the beta; no luck. I'd appreciate any advice you have. Thanks!

chris$ rake --trace (in /Users/chris/Documents/3 Web Development/3 Ruby Projects/winedark) rake aborted! undefined method``config' for nil:NilClass /opt/local/lib/ruby/gems/1.8/gems/railties-3.0.0.rc/lib/rails.rb:44:in``configuration' /opt/local/lib/ruby/gems/1.8/gems/loofah-0.4.7/lib/loofah.rb:89 /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:205:in``load_dependency' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:570:in``new_constants_in' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:205:in``load_dependency' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /opt/local/lib/ruby/gems/1.8/gems/feedzirra-0.0.23/lib/feedzirra.rb:6 /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:205:in``load_dependency' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:570:in``new_constants_in' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:205:in``load_dependency' /opt/local/lib/ruby/gems/1.8/gems/activesupport-3.0.0.rc/lib/active_support/dependencies.rb:219:in``require' /Users/chris/Documents/3 Web Development/3 Ruby Projects/winedark/config/application.rb:9 /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in``gem_original_require' /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in``require' /Users/chris/Documents/3 Web Development/3 Ruby Projects/winedark/Rakefile:4 /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2383:in``load' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2383:in``raw_load_rakefile' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2017:in 'load_rakefile' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in``standard_exception_handling' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2016:in``load_rakefile' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2000:in``run' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:2068:in``standard_exception_handling' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/lib/rake.rb:1998:in``run' /Users/chris/.gem/ruby/1.8/gems/rake-0.8.7/bin/rake:31 /opt/local/bin/rake:19:in``load' /opt/local/bin/rake:19

rails 3.0.0 final, bundler and feedzirra

Adding feedzirra in Gemfile and executing "rails s" gives me:

/usr/lib/ruby/gems/1.8/gems/railties-3.0.0/lib/rails.rb:44:in configuration': undefined methodconfig' for nil:NilClass (NoMethodError)
from /usr/lib/ruby/gems/1.8/gems/loofah-0.4.7/lib/loofah.rb:89
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:in require' from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:inrequire'
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in load_dependency' from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:591:innew_constants_in'
from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:225:in load_dependency' from /usr/lib/ruby/gems/1.8/gems/activesupport-3.0.0/lib/active_support/dependencies.rb:239:inrequire'
from /usr/lib/ruby/gems/1.8/gems/feedzirra-0.0.24/lib/feedzirra.rb:6
.
.
.

Atom:content elements which dont wrap content in CDATA blocks is stripped of any markup

Atom1: http://blogs.usask.ca/medical_education/atom.xml
Atom2: http://blog.organizedwisdom.com/health/atom.xml

The "content" elements in Atom1 have their data wrapped in CDATA blocks. The "content" elements in Atom2 do NOT.

Parsing these feeds in Feedzirra 0.0.23 (with sax-machine 0.0.15) returns content elements that has any of its HTML stripped in Atom1 - but any HTML present in Atom2 is retained.

Ideally, regardless of the presence of any CDATA blocks shouldnt the content be returned as-is?

I looked into sax_handler.rb and its cdata handler just passes it straight onto the regular characters handler:

def cdata_block(string)
  characters(string)
end

So I dont really see where the stripping is taking place. Maybe in Nokogiri?

undefined method `==' for class `ActiveSupport::BasicObject' (NameError)

/home/test/v_0.1/vendor/gems/gems/activesupport-2.3.4/lib/active_support/basic_object.rb:12:in undef_method': undefined method==' for class ActiveSupport::BasicObject' (NameError) from /home/test/v_0.1/vendor/gems/gems/activesupport-2.3.4/lib/active_support/basic_object.rb:12 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:ingem_original_require'
from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:in require' from /home/test/v_0.1/vendor/gems/gems/feedzirra-0.0.20/lib/feedzirra.rb:10 from /usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:31:ingem_original_require'

This seems like a "require "activesupport" related error. Not sure why we are seeing it all of a sudden. Are we missing something obvious?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.