A toy spider using Selenium and XPath to collect info.
Use python venv:
python3 -m venv venv
source venv/bin/activate
Or using anaconda:
conda create -n <venv_name>
conda activate <venv_name>
Run the following command.
pip3 -r requirements.txt
python3 main.py --url https://www.baidu.com \
--enable-proxy \
--skip-cloudflare-waf \
--set-driver-path resources/chromedriver
- Selenium-python
- Chrome Driver - WebDriver from Chrome
- Chrome for Testing availability
- stealth.min.js
MIT or Apache-2.0 © Leryn