michjak-szymanski / michal.szymanski.crawler Goto Github PK
View Code? Open in Web Editor NEWEasily extensible library which provides straightforward way to create Crawler-like (this one focuses rather on time limited crawling) programs in few variations (Concurrent, Normal, One-time [one page]).
Home Page: http://michal-szymanski.pl/tools/website_searcher