dangnguyentranngoc/batdongsan

BatDongSan - A pet project

The mainly target of this project is learn and practice:

• Go language 🔰
• Crawl web data 🔰
• ETL 🔰
• Data Lake 🔰
• Data Warehouse 🔰
• Data Analysis 🔰

DESCRIPTION

The online websites are contain a huge information covering all the aspect of our lives. And Web Mining is one of the most important and oldest task of the Data Mining tasks. In this mini project I want to research and apply the Web Mining techniques to extract information from a real estate website. For the study porpose, I will use the "handy tools" built from Go and Python languages, in conjunction with Hadoop, PostgreSQL, .... Hope that on the ver 2.0 the project will be runed on Docker and Cloud.

THE ARCHITECTURE

I want a picture 🎞 here. But I didn't draw it. So please use your imagination 🤗.

COLLECT DATA

Crawling data and store somewhere on the Earth 🌏

PROCESSING DATA - ETL

Data WareHouse here 🏭 !

DATA ANALYTICS - REPORT

📈📉📊