gonali Goto Github PK

bazhuayudatadownloader

bazhuayurulers

bazhuayuxmldatatomysql

bootstrap

The most popular HTML, CSS, and JavaScript framework for developing responsive, mobile first projects on the web.

crawlerscheduler

crawlerwebmagic

distribute_crawler

使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

distributedcrawler

华南理工大学高英实验室进行的分布式爬虫项目,除了实验室内部人员外,不得私自传播.

elasticsearch-hq

Monitoring and Management Web Application for ElasticSearch instances and clusters.

jsoncpp

A C++ library for interacting with JSON.

jsoup

jsoup: Java HTML Parser, with best of DOM, CSS, and jquery

libxml2

linux

Linux kernel source tree

open-source-search-engine

A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.

qiaokebao

redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, HyperLogLogs, Bitmaps.

scrapy-redis

Redis-based components for scrapy that allows distributed crawling

tinyxml2

TinyXML2 is a simple, small, efficient, C++ XML parser that can be easily integrating into other programs.

webmagic

A scalable web crawler framework.

wifiauthserver

zhihucrawler

event-driven crawler implemented by C++

zkcrawler