ethanwtl/climatechange

Automate ML with Detecting Climate Changes

Understand Climate Types using Unsupervised Machine Learning

Introduction:

Hi, Scientists and Geologists. 😃

We are working on understanding different climate types in North American Continent.

This research has a focus on using unsupervised learning like Kmean and Self Organizing Maps to clustering different climate types.

Background:

Datasets: We received 10 years of Pressure and Wind data for North American Continent in csv format.

Region of Interests: As the contour lines moves as time goes by, we can see they form areas in a circle that we interested in.

Climate types: There are four different types of climate types based on the pressure mapping in ROI.

climate type	Description
COL	a weather system that has become detached or "cut off" from the main jet stream
CL	really similar to COL, but a little bit different in how contour lines mapped
COH	areas of relatively high atmospheric pressure compared to their surroundings
NROI	Non Region of Interet

Goal: Try to determine different climate type from pressure and wind maps

---

ROI Extraction:

To increase analysis efficiency, we will only focus on Region Of Interests.

• Switching csv pressure map into grey scale map.
• Apply Connected Component to seprated out different level of cluster.
• Deploy Open-CV to detect the circle cluster.
• Consulting with experts in enviornmental field to label the ROI for us.

Data Engineering:

After cutting ROI from the entire pressure map, we performed a series of data engineering techniques.

1. Normalization then cut: Considering the climate type determined by the difference between middle circle and boarder value, we normalize the entire pressure map and then cropping out the 15*15 ROI.

2. Cut then Normalization: We cut the ROI out then normalize on the 15*15 image.

3. Wind map: Wind directions are presented as positive and negative values, normalization then crop.

Unsupervised Machine Learning:

K-mean: we first explore the possibility using K-mean to clustering 4 different climate types.

• Clusters: 4
• Acc: 41%

Self Organizing Maps: we delopy SOMs for clustering using nerual architecture

• Size: 10*10
• m_distance: 4
• Leanring_rate: 0.05
• steps: 75000
• ACC: 86%
• Confusion Matrix:

	CL	COH	COL	NROI
CL	164	0	14	0
COH	2	156	2	1
COL	40	1	276	10
NROI	0	3	30	38

Important analysis

The following diagram present the error rate within the SOM. We include the actual labels that fall in each neuron in SOM. We can clearly see that along the edge between Green and Orange, which represent between CL and COL, the error rate is increasing.

Roadmap

• ROI Extraction and Labeling
• Label Validation
• Data Engineering
• Unsupervised ML
  • Kmean
  • SOMs
• Supervised ML
  • CNN
  • RNN

Contact

Ethan Wang - [email protected] - Linkedin Profile

Project Link: https://github.com/matsudatakeshi27/HeartDiseasePakula

Ackowledgment:

• The self-organizing map
• Review of the self-organizing map (SOM) approach in water resources: Analysis, modelling and application
• A Review of K-mean Algorithm