Hi, Scientists and Geologists. ๐
We are working on understanding different climate types in North American Continent.
This research has a focus on using unsupervised learning like Kmean and Self Organizing Maps to clustering different climate types.
Datasets:
We received 10 years of Pressure and Wind data for North American Continent in csv format.
Region of Interests:
As the contour lines moves as time goes by, we can see they form areas in a circle that we interested in.
Climate types:
There are four different types of climate types based on the pressure mapping in ROI.
climate type | Description |
---|---|
COL | a weather system that has become detached or "cut off" from the main jet stream |
CL | really similar to COL, but a little bit different in how contour lines mapped |
COH | areas of relatively high atmospheric pressure compared to their surroundings |
NROI | Non Region of Interet |
Goal:
Try to determine different climate type from pressure and wind maps
To increase analysis efficiency, we will only focus on Region Of Interests.
- Switching csv pressure map into grey scale map.
- Apply Connected Component to seprated out different level of cluster.
- Deploy Open-CV to detect the circle cluster.
- Consulting with experts in enviornmental field to label the ROI for us.
![](https://private-user-images.githubusercontent.com/97998419/250327076-11da3fd7-b657-4f51-85e1-a4a43e91a7e8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwMzUwMjIsIm5iZiI6MTcyNDAzNDcyMiwicGF0aCI6Ii85Nzk5ODQxOS8yNTAzMjcwNzYtMTFkYTNmZDctYjY1Ny00ZjUxLTg1ZTEtYTRhNDNlOTFhN2U4LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDAyMzIwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWEzMjNlZTEzMmZiNDU3NWQzZGMyNjIyNjQ0NWM2YjhiMDkxZWYxOGI3MWQyYWQ5ZGIyNmZlYWMzNGEzNzUxNTUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.4EUeDG5sW4nsB64zu5DecapnZ0uV-_vxj2gXeafAs2g)
![](https://private-user-images.githubusercontent.com/97998419/250327497-19c25973-de73-4fb0-9d91-43e923bc9c5a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwMzUwMjIsIm5iZiI6MTcyNDAzNDcyMiwicGF0aCI6Ii85Nzk5ODQxOS8yNTAzMjc0OTctMTljMjU5NzMtZGU3My00ZmIwLTlkOTEtNDNlOTIzYmM5YzVhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDAyMzIwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVkMzA4MTk2MTI4MTUyODIxZjA3YmQ3Yjg0YWUyMTk0ZGQxYjVhZjVhNTNiMTMzZTZlNGQwOTI1ZmU4OTY5NGImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.ip8bzpuO_SNlOVnr7XiVpritMTiU3oqmNEu5zsrJ8ho)
![](https://private-user-images.githubusercontent.com/97998419/250323019-2b1772de-3b7c-4115-bb92-81437d681fd7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwMzUwMjIsIm5iZiI6MTcyNDAzNDcyMiwicGF0aCI6Ii85Nzk5ODQxOS8yNTAzMjMwMTktMmIxNzcyZGUtM2I3Yy00MTE1LWJiOTItODE0MzdkNjgxZmQ3LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDAyMzIwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTk4Zjg0YzI1MTUxNGVlZTg1MWUyMjA2NzZmMWM4OWZjNjg1NzZmM2RjMDExMGZjODAwMjQ4YmUzMWRkMjRhZGMmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.CVBhXn9X290ttELX8Zp0IKIEp_JPFb2sUIOBwJ4rWzQ)
![](https://private-user-images.githubusercontent.com/97998419/250323365-52548b8b-20fb-4feb-9499-9161e93cc090.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQwMzUwMjIsIm5iZiI6MTcyNDAzNDcyMiwicGF0aCI6Ii85Nzk5ODQxOS8yNTAzMjMzNjUtNTI1NDhiOGItMjBmYi00ZmViLTk0OTktOTE2MWU5M2NjMDkwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTklMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE5VDAyMzIwMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWRhMmRjOTNkZmRhMzA5ZDQ2NGRmNmRiNDliOTM2ODdmYjY0MTk1MDI4N2M0M2ZkYTYwNzk2ODM0NDZkNTk2MTAmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.SyhD0h7ydZPPTr_1F8cZ2-lEJrWhBNOzOVqpS0_hdXA)
After cutting ROI from the entire pressure map, we performed a series of data engineering techniques.
-
Normalization then cut:
Considering the climate type determined by the difference between middle circle and boarder value, we normalize the entire pressure map and then cropping out the 15*15 ROI. -
Cut then Normalization:
We cut the ROI out then normalize on the 15*15 image. -
Wind map:
Wind directions are presented as positive and negative values, normalization then crop.
K-mean: we first explore the possibility using K-mean to clustering 4 different climate types.
Clusters:
4Acc:
41%
Self Organizing Maps: we delopy SOMs for clustering using nerual architecture
Size:
10*10m_distance:
4Leanring_rate:
0.05steps:
75000ACC:
86%Confusion Matrix:
CL | COH | COL | NROI | |
---|---|---|---|---|
CL | 164 | 0 | 14 | 0 |
COH | 2 | 156 | 2 | 1 |
COL | 40 | 1 | 276 | 10 |
NROI | 0 | 3 | 30 | 38 |
The following diagram present the error rate within the SOM. We include the actual labels that fall in each neuron in SOM. We can clearly see that along the edge between Green and Orange, which represent between CL and COL, the error rate is increasing.
- ROI Extraction and Labeling
- Label Validation
- Data Engineering
- Unsupervised ML
- Kmean
- SOMs
- Supervised ML
- CNN
- RNN
Ethan Wang - [email protected] - Linkedin Profile
Project Link: https://github.com/matsudatakeshi27/HeartDiseasePakula