harukasan/fluent-plugin-rss

Fluent plugin to fetch RSS feed

harukasan.jp

The new harukasan.jp

fluent-plugin-rss's Issues

Noob configuration guidance please

I found your plugin but am struggling (I think) to configure it correctly.

Installed (successfully?)

fluent-gem install fluentd-plugin-rss

Fetching: fluent-plugin-rss-0.0.1.gem (100%)
Successfully installed fluent-plugin-rss-0.0.1
Parsing documentation for fluent-plugin-rss-0.0.1
Installing ri documentation for fluent-plugin-rss-0.0.1
Done installing documentation for fluent-plugin-rss after 0 seconds
1 gem installed

fluent-gem list

*** LOCAL GEMS ***
...
fluent-plugin-rss (0.0.1)
...

Configured (unclear)

Currently have both rss.conf and fluent-plugin-rss.conf

The 'type' property was not referenced but I think it should be 'rss'? Looking at your source (not a Ruby developer), it appears the 'tag' is to identify the emitted data so could be anything. I've tried various permutations of this:

<source>
  type rss
  tag cnn
  url http://rss.cnn.com/rss/cnn_latest.rss
  attrs date, title
</source>

Reviewing your code, it appears the RSS feed must contain a 'date' element. And I suspected this was the problem with the 1st feed that I tried which contained no 'date'. I extracted the RSS parsing and enumeration from your plugin (below). I've tried 2 other feeds that both use 'pubDate' (http://lorem-rss.herokuapp.com/feed; http://rss.cnn.com/rss/cnn_latest.rss) and these both work; they're 'put' and I assume would be emitted too.

#!/usr/bin/env ruby
require 'rss'

@tag = 'test'
@current_time = Time.new(2016,6,15,0,0,0, "+09:00").utc
@url = 'http://rss.cnn.com/rss/cnn_latest.rss'

next_current_time = @current_time

rss = RSS::Parser.parse(@url)

rss.items.each do |item|
    time = Time.parse item.date.to_s
    if time > @current_time
        puts "#{@tag} --- #{Time.parse(item.date.to_s)} --- #{item.title}"
        next_current_time = time if time > next_current_time
    end
end

I see no evident output :-(

So I'm doing something wrong. Any help would be appreciated.

rss fluent in error with td-agent v2

Hi,
i'm trying to use your rss plugin, but seems doesn't work at all.
steps to reproduce are:

1. install td-agent v2
2. install plugin with command td-agent-gem install fluent-plugin-rss
3. add in td-agent.conf

<source>
	@type rss
	tag rss.nytimes                                     
	url http://example.com/input-your-rss-feed-here.rdf 
	interval 10m                                        
	attrs date, title                                   
</source>

start td-agent
after start, the service crash with error:

/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-rss-0.0.1/lib/fluent/plugin/in_rss.rb:4:in `<module:Fluent>': uninitialized constant Fluent::Input (NameError)
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluent-plugin-rss-0.0.1/lib/fluent/plugin/in_rss.rb:3:in `<top (required)>'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/registry.rb:102:in `block in search'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/registry.rb:99:in `each'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/registry.rb:99:in `search'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/registry.rb:44:in `lookup'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/plugin.rb:146:in `new_impl'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/plugin.rb:100:in `new_input'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/root_agent.rb:277:in `add_source'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/root_agent.rb:122:in `block in configure'
        from /opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems/fluentd-1.1.3/lib/fluent/root_agent.rb:118:in `each'

could you please help me?

I'm sorry. I mistook your project for another project, please ignore me.