pirj/tor-privoxy

Tor-privoxy is a Ruby Mechanize wrapper to access the web with mechanize via Tor/Privoxy It allows to use multiple Privoxy instances, switch endpoints, switch proxy when you get 4xx HTTP code Useful for web robots, scanners, grabbers when accessing sites which may ban/block you unexpectedly

Tor/privoxy wrapped Mechanize

tor-privoxy is a Ruby Mechanize wrapper for accessing the web via Tor/Privoxy. It allows multiple Privoxy instances, switching endpoints, and switching the proxy when you get an HTTP 4xx error code. It is useful for web robots, scanners, and scrapers when accessing sites which may ban/block you unexpectedly

Using

The first step is to install the gem:

gem install tor-privoxy

To use in your application:

require 'tor-privoxy'

To get a Mechanize instance wrapped to use Tor and able to use another endpoint when it encounters an HTTP 4xx code:

agent = TorPrivoxy::Agent.new '127.0.0.1', '', {8118 => 9051} do |agent|
  sleep 1
  puts "New IP is #{agent.ip}"
end

Passing block is optional:

agent = TorPrivoxy::Agent.new '127.0.0.1', '', 8118 => 9051

First parameter is a proxy host, second is Tor control port password:

agent = TorPrivoxy::Agent.new '127.0.0.1', '', 8118 => 9051

The hash is in format proxyport => torcontrolport. Yes, you may provide as many as you want, but I don't have an idea why I initially did it like so.

And use the agent as a usual Mechanize agent instance:

agent.get "http://example.com"

Configuration options

Configuration options are passed when creating an agent and consist of:

• IP/Host of machine where Tor/Privoxy resides
• password for Tor Control
• a hash of Privoxy port => Tor port
• a block which is called when agent switches to a new endpoint

Author

Created by Phil Pirozhkov

Origin

Future

• No Mechanize dependency, ability to work with any HTTP library
• Extend configuration options, allowing for fine proxy setting control
• Better "ban" detection, e.g. Captcha, etc.

tor-privoxy's Issues

stack level too deep

Hey,

Great job with this gem.
After using for longer period, after couple 403 responses, the gem raises stack level too deep error. Is there any possibility to fix it?

Connection Refused

Hello, how are you friend?

First of all, thanks for your brilliant gem. Well, I really loved your proposal, but I'm having some problems. I try to make a test "Rake Task", exactly as following:

task test: :environment do
    require 'tor-privoxy'
    agent ||= TorPrivoxy::Agent.new '127.0.0.1', '', {8123 => 9051} do |agent|
      sleep 10
      puts "New IP is #{agent.ip}"
    end
end

And I'm getting the following error on Terminal:

rake aborted!
connection refused: 127.0.0.1:8123
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/net-http-persistent-2.9/lib/net/http/persistent.rb:640:in `rescue in connection_for'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/net-http-persistent-2.9/lib/net/http/persistent.rb:589:in `connection_for'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/net-http-persistent-2.9/lib/net/http/persistent.rb:981:in `request'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/mechanize-2.6.0/lib/mechanize/http/agent.rb:263:in `fetch'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/mechanize-2.6.0/lib/mechanize.rb:431:in `get'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:38:in `ip'
/home/paladini/Documents/Projetos/earn_litecoin/lib/tasks/hack.rake:9:in `block (3 levels) in <top (required)>'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:in `call'
/home/paladini/.rvm/gems/ruby-2.0.0-p353@earn_litecoin/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:in `initialize'
/home/paladini/Documents/Projetos/earn_litecoin/lib/tasks/hack.rake:7:in `new'
/home/paladini/Documents/Projetos/earn_litecoin/lib/tasks/hack.rake:7:in `block (2 levels) in <top (required)>'

And one thing that I doesn't understand in your gem: only this code can provide me access to a website, changing IP Address every time? (stopping issues like blocks and bans). It's like a Proxy changer?

Cannot load file

Getting this:

2.1.1 :001 > require 'tor-privoxy' LoadError: cannot load such file -- tor-privoxy from /home/tristan/.rvm/gems/ruby-2.1.1/gems/activesupport-4.0.4/lib/active_support/dependencies.rb:229:inrequire'
from /home/tristan/.rvm/gems/ruby-2.1.1/gems/activesupport-4.0.4/lib/active_support/dependencies.rb:229:in block in require' from /home/tristan/.rvm/gems/ruby-2.1.1/gems/activesupport-4.0.4/lib/active_support/dependencies.rb:214:inload_dependency'
from /home/tristan/.rvm/gems/ruby-2.1.1/gems/activesupport-4.0.4/lib/active_support/dependencies.rb:229:in require' from (irb):1 from /home/tristan/.rvm/gems/ruby-2.1.1/gems/railties-4.0.4/lib/rails/commands/console.rb:90:instart'
from /home/tristan/.rvm/gems/ruby-2.1.1/gems/railties-4.0.4/lib/rails/commands/console.rb:9:in start' from /home/tristan/.rvm/gems/ruby-2.1.1/gems/railties-4.0.4/lib/rails/commands.rb:62:in<top (required)>'
from bin/rails:4:in require' from bin/rails:4:in

'`

Error 500 on mechanize

Hi, i'm having an error on mechanize when gets the ip from http://ifconfig.me/ip

This is the full error:

/usr/local/rvm/gems/ruby-2.1.1/gems/mechanize-2.7.3/lib/mechanize/http/agent.rb:308:in fetch': 500 => Net::HTTPInternalServerError for http://ifconfig.me/ip -- unhandled response (Mechanize::ResponseCodeError) from /usr/local/rvm/gems/ruby-2.1.1/gems/mechanize-2.7.3/lib/mechanize.rb:440:inget'
from /usr/local/rvm/gems/ruby-2.1.1/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:38:in ip' from test.rb:8:inblock in

'
from /usr/local/rvm/gems/ruby-2.1.1/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:in call' from /usr/local/rvm/gems/ruby-2.1.1/gems/tor-privoxy-0.1.1/lib/tor-privoxy/agent.rb:11:ininitialize'
from test.rb:5:in new' from test.rb:5:in'

The ruby script is the same as the example:

require 'tor-privoxy'

agent ||= TorPrivoxy::Agent.new '127.0.0.1', '', {8123 => 9051} do |agent|
sleep 10
puts "New IP is #{agent.ip}"
end

Sometimes works and get the correct ip, but then got that error, and idea?

Regards,