faris-abukhader/mawdoo3-scrapper

Mawdoo3 Arabic website scrapper using python

Mawdoo3 scrapper

📔 بالعربي 📔

🚩 Table of Contents

• Introduction
• Installation
• Development setup
• Packages
• License

Introduction

Warning: This project is only for study purpose , please don’t re-share these articles under your name , all these articles is only belongs to Mawdoo3 .

how to get 144k articles and more . . . ?!

• first create object from class mawdoo3
• second call function save_all_articles_title_into_file
• finally take the name of generated file from previous step and pass it to save_all_articles_into_file

Installation

🔘 Cloning repository

1. On GitHub.com, navigate to the main page of the repository.
2. Above the list of files, click Code.
3. Copy the URL for the repository.
4. Open Terminal.
5. Change the current working directory to the location where you want the cloned directory.
6. Type git clone, and then paste the URL you copied earlier.

git clone github.com/Faris-abukhader/mawdoo3-scrapper

Press Enter to create your local clone

git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY
> Cloning into `mawdoo3-scrapper`...
> remote: Counting objects: 10, done.
> remote: Compressing objects: 100% (8/8), done.
> remove: Total 10 (delta 1), reused 10 (delta 1)
> Unpacking objects: 100% (10/10), done.

Development setup

To set up this project you need to download Python in your machine or if you have it make sure you have the latest version of it.

🔘 Checking up Python version in mac

python3 -V

🔘 Checking up Python version in windows

python --version

🔘 Downloading Python

for Windows

Download the windows installer from Python offical website make sure you have download the latest version of Python.

for Mac

• You can download Python using brew CLI

brew install python

• You can download Python mac version through the offical website

---

🔘 Downloading the packages

Go to project direct where requirements.txt is exist and type in terminal :

pip install -r requirements.txt 

---

📦 Packages

Name	Description
BeautifulSoup	Beautiful Soup is a Python library for pulling data out of HTML and XML files.
selenium	The selenium package is used to automate web browser interaction from Python.
aiohttp	Asynchronous HTTP Client/Server for asyncio and Python.
asyncio	asyncio is a library to write concurrent code using the async/await syntax.

📜 License

This software is licensed under the MIT © FaRiS.