mawada-sweis/clustering-analysis

Clustering Analysis is used to analyze the travel behaviors, preferences, and attitudes of New York City citizens.

Hi 👋, I'm Mawadda

A passionate AI Engineer 🤖 and Backend developer 👨‍💻

• 🔭 I’m currently working on Smart task management system
• 👯 Proud of the machine learning project Predict car price
• 🌱 I’m currently learning Power BI, Docker, and other tools.
• 💬 Ask me about Machine learning, Data science, and Project management.

My Skill Set

Frontend

Frameworks and Packages

Backend

DevOps

Github Stats

clustering-analysis's Issues

Discuss result

Meet to discuss the clustering results for various algorithms and our progress.

skeleton folder

Having a well-organized project structure makes it easy to understand and make changes.
In general, these are the basic folders we will work on:

data, models, notebooks, utils and src folders.

EDA

• The plot displays the percentage of missing data (before and after Cleaning).
• Plotting displays the column value counts.
• Correlation graph.
• Numeric column boxplot.
• Plots of distribution.
• Write Insights and conclusions

Transformation

Do Feature Engineering includes:

• One hot encoding on categorical features (Nominal).
• Extract the day, month and year features from each date feature.

Explore project & dataset (Goal, features, data)

This task is for everyone.
We will meet to identify the purpose of our project and to study the features and their potential values (data) in order to determine how to deal with them.

Grouping Data

Group the data based on the household feature to better understand the data.

Hierarchal model

• Choose the best threshold value.
• Display the dendrogram graph.

Cleaning dataset

Cleaning the data by removing columns that are unnecessary, or contain a large amount of missing data. in addition to dealing with noisy or inconsistent data.

Scaling dataset

• share final two data sets scaled with:
• • standard scaler
• • min max scaler

Explore Project file 4

understand then give a summary of the fourth link in the data website.

K-mean model

• Choose the best K value.
• Display the Elbow graph.
• How to Display the clustering?

Basic Dataset information

Provide the basic information about the dataset in the notebook file. It must contain:

• Dataset Statistical Describing.
• Dataset information.
• Display the percentage of missing/noisy/inconsistent data for each column.
• Suggest method handling based on the data.

Share final dataset

Based on the dataset and our sumption and understanding:

• What columns we will have to keep?
• How to deal with samples of other persons in the same household?
• How to deal with several type of collected data method in the same dataset?

Output expected:
Dataset with specific collected data method and with only persons who fill out the servay.

Explore Project file 1

understand then give a summary of the first link in the data website.

DBSCAN model

Readme file

The Citywide Mobility Survey (CMS)

The Citywide Mobility Survey (CMS) is a survey conducted by the New York City Department of Transportation (DOT) to gather information about the travel behavior, preferences, and attitudes of New York City residents. The survey is conducted periodically, and the data collected is used to inform transportation planning and policy decisions.

Table of Contents

• Data Objectives

• Project Objectives

Data Objectives

The primary objectives of the CMS data collection are:

• Identify the factors and experiences that drive transportation choices for New York City residents.
• Understand current views on the state of transportation within the City.
• Measure attitudes toward current transportation issues and topics in New York City.

Project Objective

The objective of this project is to analyze the CMS dataset to gain insights into the travel behavior, preferences, and attitudes of New York City residents. Specifically, we aim to:

• Understand the relationship between residents' behaviors, preferences, attitudes, and traveling methods.
• Identify any trends or patterns in the data that may be relevant to transportation planning and policy decisions.

By achieving these objectives, we hope to contribute to the ongoing efforts to improve transportation in New York City and enhance the mobility of its residents.