Install all dependencies:
python setup.py develop
Configure environment variables:
export MONGO_URI='mongodb://<dbuser>:<dbpassword>@<host>:<port>/<db>' (optional) export HTTP_PROXY='http://localhost:1234' export SPIDER_URLS_FILE='/home/user/spider_urls.txt'
Run crawler:
$ run_crawler
bogdal / price-tracker Goto Github PK
View Code? Open in Web Editor NEWCeneo.pl Price tracker
License: BSD 2-Clause "Simplified" License