It extract homepage text
-
Install modules
pip3 install -r requirements.txt
-
python3 crawler.py --help
-
Provide all input arguments mentioned in help section For Example:
python3 crawler.py --input_file text_input.csv --output_file testoutput --nprocesses 10 --website_column website --parser BeautifulSoup Note: Input file should contain website column.
> NA
> Aakash Kag