devoeop/e-commerce-retail-data-analytics

End-to-end design and implementation of a comprehensive data platform for retail analytics in an e-commerce organization.

E-commerce-Retail-Data-Analytics

• End-to-end design and implementation of a comprehensive data platform for retail analytics in an e-commerce organization integrating MySQL for OLTP, MongoDB for product catalog data, PostgreSQL for staging, and production data warehousing.

• Designed and deployed a star schema data warehouse named "softcart," incorporating dimension tables (e.g., softcartDimDate, softcartDimCategory, softcartDimCountry) and a fact table (softcartFactSales), enabling seamless reporting and analytics on yearly, monthly, weekly, and daily bases.

• Implemented ETL pipelines powered by Apache Airflow to automate the extraction, transformation, and loading of incremental data between OLTP, NoSQL (MongoDB), and data warehouse instances, ensuring data consistency and freshness.

• Implemented a Business Intelligence dashboard using Cognos, featuring dynamic visualizations such as quarterly sales bar charts, electronic goods sales pie charts by category, and monthly total sales line charts for enhanced business insights.

• Utilized Hadoop and Spark as the big data platform for collecting and analyzing large datasets, enhancing SoftCart's analytical capabilities.

• Databases: MySQL (OLTP), MongoDB (NoSQL), PostgreSQL (Staging and Production)

• Big Data Technologies: Hadoop Cluster, Spark for analytics

• ETL Tools: Apache Airflow for orchestrating ETL pipelines

• Business Intelligence: Cognos for creating dynamic and interactive dashboards

• Data Warehousing: Designed star schema, implemented staging environment, and loaded transformed data into the data warehouse using CSV files.