elecnix/evalmtl

Téléchargement des évaluations foncières de Montréal

evalmtl's Issues

Incomplete scrape

I get the following errors after scrape-evalweb.rb. Suggestions?

C:/Users/me/evalmtl-master/lib/scraper.rb:29:in rescue in perform_request': undefined methodpage' for #<TypeError: Fixnum is not a String> (NoMethodError)
from C:/Users/me/evalmtl-master/lib/scraper.rb:17:in perform_request' from C:/Users/me/evalmtl-master/lib/scraper.rb:40:insearch_street'
from C:/Users/me/evalmtl-master/lib/scraper.rb:72:in search_street' from C:/Users/me/evalmtl-master/lib/scraper.rb:124:inblock in scrape'
from C:/Users/me/evalmtl-master/lib/scraper.rb:119:in each' from C:/Users/me/evalmtl-master/lib/scraper.rb:119:inscrape'
from scrape-evalweb.rb:5:in `

'

Évaluations foncières de Montréal, rôle de 2014

http://bit.ly/19axHQ9

storage in cache directory is very slow

scrape-evalweb.rb writes thousands of HTML files to a "cache" directory, tar czf evalweb-cache.tgz takes a long time, and is only a sloppy work-around.

Instead, a real database should be used, such as sqlite or a berkeley db, allowing fast random read/writes, but also fast sequencial reads.