Bablo (Бабло) is a web scraping tool designed to explore and collect URLs from a specified starting URL. It utilizes the "github.com/gocolly/colly/v2" package.
- Recursive crawling
- Multithreading
- Clone or Download: Clone this repository or download the code to your local machine.
- Install Golang: Install the latest Golang from https://go.dev/dl/. For example:
wget https://go.dev/dl/go1.21.4.linux-amd64.tar.gz rm -rf /usr/local/go && tar -C /usr/local -xzf go1.21.4.linux-amd64.tar.gz export PATH=$PATH:/usr/local/go/bin go version
- Install colly: Run the following commands:
go mod init bablo go mod tidy go get github.com/gocolly/colly/v2@latest
- Run the tool: Run the tool using the following command:
go run main.go -t <thread-count> https://example.com
- Automation: For automating this tool with others check - https://github.com/grozdniyandy/hackerlines
You can either check the "Usage" or download already compiled code from "releases".
This code uses the Go standard library and one external dependency github.com/gocolly/colly/v2.
This code is released under the MIT License.
Bablo is developed by GrozdniyAndy of XSS.is.
Feel free to contribute, report issues, or suggest improvements by creating pull requests or issues in the GitHub repository. Enjoy using this crawler!