hueristiq / xcrawl3r Goto Github PK
View Code? Open in Web Editor NEWA command-line interface (CLI) based utility to recursively crawl webpages. It is designed to systematically browse webpages' URLs and follow links to discover linked webpages' URLs.
Home Page: https://github.com/hueristiq/xcrawl3r
License: MIT License