Detecting the presence of an Electic Vehicle that is connected to the grid.
This project utilizes an open data set from GridCure. The intention of this project is the following:
- Interesting aspects about this dataset
- Identify which households own an Electric Vehicle(EV)
- Determine the probability that an EV is charging at any given interval
I am going to tackle this one first because it affected how I treated cleaning and ultimately how I trained my algorithms on it.
- Using findNaN function that I wrote, I discovered that there were four households that containted null data in the training set and two households in the test set. Given that these houses that had null data only comprimised less than 0.5% of the data I felt comfortable eliminating them from the datasets. It appears that these null values comprise an entire months worth of data. In all likely hood these occupants were either moving in or moving out their houses... thus this dataset only captured their first month or last month of electricity usage.
- Using the average usage of each household as a representation for the training set, it appears over the course of two weeks that each day there is spike in electricity. Graph below.
Filtering down into a single days worth of electicity usage, it becomes apparent that usage begins to ramp up in the early morning, remains steady throughout mid morning and early afternoon, peaks in the evening and then finally falls off precipitously right after peak usage. This makes intuitive sense, there is likely minimal electricity use when everyone in a house is asleep and usage will start to increase as people wake up and start to make breakfast,use the toaster, stove, etc... and electricity use will be at it's peak in the evening when everyone in the household is home and using all their computers, TV's,running the air conditioner, and charging their EV if they own one. Graph below.
Approximately 1/3 of the training set households owned an EV, or 485/1586 = 30.5% if you prefer decimals. The House ID can be found in the the pickled file 'Houses_that_own_ev.pkl' located in this repo's data_files folder. Or just go to the link below.
Now, over any one interval it appears that at most 16% of households have an EV plugged in and given that almost two sigmas worth of the households has their EV plugged in less than 10 % of the time... this problem appears to be a minority class detection problem. Very similar to a fraud detection problem, I needed to detect with high recall an occurrence that only happens a small percentage of the time. I used a variety a techniques ranging from using the synthetic minority oversampling technique(SMOTE) to a gradient boosted tree classifier. Yet no matter what technique or combination of techniques I still had a very low recall. That is not good to say the least. Enter Neural Networks and the Long Short Term Memory(LSTM) layer strucutre.
I did not have the answer if a EV was charging or not on the test data, given that it is test data. I surely could split up my training data though, for which I do know when EVs were charging or not. By treating this dataset as a time series and training an LSTM Neural Network on the train set, I was able to achieve a recall above 75% on some of the households in the training data. This result is far better than the less than 10% recall I was getting with other techniques.
Graph below:
- Blue is training data and red is test data
- As you can see the neural net captures the pattern including the spikes in usage very well
Finally I trained my Neural Net on the average households usage over a 30 min interval. The probability that each a household in the test data owns an EV can be found in the pickled file found here.
https://github.com/ajduncan3/Electric_Vehicle_Detection/blob/master/data_files/predicted_kwh_probabilities.pkl
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent