We introduce physics-informed neural netwroks for prognostics, which is the engineering discipline of predicting the time in which a system will fail. Centred on predicting the Remaining Useful Life through Health Index formulation, this paper presents a Physics-Based Machine Learning prognostics framework that imposes knowledge of the system's Health Index as loss functions. A case implementation on experimental information from 8 induction engines was conducted. This paper carefully assesses the performance of the physics-based machine learning model by comparing the predictions with available experimental data. The investigations reveal that the physics-based machine learning model can accurately improve the Remaining Useful Life predictions by more than 30 percent as compared to direct methods. Owing to the additional knowledge, less labelled training data is required, and the predictions are scientifically consistent with the known knowledge. The foray of physics-based machine learning to prognostics evidences the great potential of its application for broader applications in prognostics.
Implementation using Tensorflow 2.0 can be accessed here:
Implementation using Tensorflow 1.0 can be accessed here:
Full Final Year Report can be accessed here:
In general, there are a series of steps taken to predicting the remaining useful life of a system.
The first step is data collection and pre-processing where raw data is cleaned to ensure its accuracy, completeness and consistency. Raw sensor data were collected by Professor Wesley and his team from the University of Tennessee. They conducted thermal ageing on eight induction motors and collected the following 11 channels of raw signals.
- 3 phases of current (Current 1, 2, 3)
- 3 phases of voltage (Voltage 1, 2, 3)
- 2 accelerometer signal (Vibration 1, 2)
- Tachometer signal (motor speed)
- Thermocouple signal (temperature)
- Acoustic sensor signal (sound)
The second step is feature engineering which can be broken down into two sub steps, mainly (1) feature extraction which converts the pre-processed raw data into features which can explain the data better and (2) feature selection which ranks the features based on its distinctiveness and extracts the most important features that are relevant for the problem. The pre-processing and feature extraction stages were conducted by Yang Feng and his team by substituting the missing values with values from the previous cycle and applying 11 statistical indicators as shown in the table to each of the 11 channels of raw data to obtain 121 features.
Having the features created, I proceeded to do feature selection to rank 121 features based on its distinctiveness and extracted the 20 most important features. The method used was fishers ratio which measures the discriminating power between a healthy class and an unhealthy class. We assumed that the motor was healthy in the first four cycles and unhealthy in the last four cycles. Essentially, the larger the ratio, the more the feature has changed when the motor degrades from a healthy to an unhealthy state, which makes the feature more distinctive.
The last step is to use the features to predict the remaining useful life. In this project, prediction was achieved through a two-stage modelling process with health index as an intermediary. The health index represents the state of health of the system. In the first stage, the feature will be used as input into the physics-based machine learning framework to predict the Health Index (HI) of the system. This framework will be the main contribution of this project and it was build using tensorflow 2.0. In the second stage, the health index is mapped to the predicted RUL, after which the final RUL prediction is produced through an ensemble approach.
This proposed physics-based machine learning model will incorporate knowledge through the following 3 rules of the health index:
- Predicted Health Index value for the subsequent cycle cannot increase
- Deviation of the predicted Health Index value within each cycle should be small
- Drop in the predicted Health Index for the subsequent cycle is small.
The first rule is that the predicted health index for subsequent cycles cannot increase (this means that the health of the system cannot improve overtime) To enforce this rule into the neural network we add Loss Rule 1 as a penalty term into the loss function. How this loss term works is if the mean health index for cycle i+1 is more than the mean health index for cycle i, this indicates that the health index increases and the term in the bracket will be positive.
When passed through the ReLu function, the output will be reflected linearly, and the optimisation process will attempt to reduce this term until it is close to zero. When it does so, the mean predicted health index does not increase anymore. In the event that the mean health index already decreases, the term in the ReLu function will be negative and the loss term will be zero, hence physical consistency is achieved.
The second rule is that all the predicted health index for the same cycle should be similar. To enforce this rule into the neural network we add Loss Rule 2 as a penalty term into the loss function How this loss term works is that it is equal to the mean absolute percentage error between the health index predictions within a cycle and their average value. Essentially, a larger mean absolute percentage error will reflect a larger deviation of the health index within the cycle. The optimisation process will attempt to reduce this term until it is close to zero hence forcing this physical constraint of a small deviation within each cycle.
Where t represents the data number within a cycle, n represents the last data set in the cycle.
For the last rule, we want to limit the drop in the health index by 20 percent. In other words, from one cycle to the next, we do not expect the health of the system to degrade too drastically. As seen from the inequality formula, if the drop in the mean health index is more than 20 percent, we can bring over the right-hand side term to obtain the following. If this term is positive, it means that the drop is more than 20 percent and when we pass it through the ReLu function, the output will be positive too. The optimisation process will attempt to reduce this term until it is close to zero. When it does so, the mean predicted health index should not decrease by more than 20 percent.
Moving on to the boundary conditions, we take the first cycle to be healthy, hence it will have a health index of 100% and the last cycle to be unhealthy, hence it will have a health index of 10%. The loss term will be mean square error between the predicted value and true value for the first and last cycle.
Ultimately, we can obtain our overall loss function by adding all the loss terms as shown in this framework. The 20 most distinctive features and cycles are input to predict the health index which should then be constrained by the three rules and the boundary conditions
Using this physics based machine learning framework, we used the 20 features and cycles from each of the 8 engines to train 8 separate health index prediction models. When we ran the features of each engine through their corresponding model. We found that all the engines show a similar degradation trend which illustrates (1) a gradual decrease in health index, (2) the small deviation in Health Index for the same cycle and (3) the boundary conditions are met where the first cycle is 100% healthy and last cycle is 10% healthy. So essentially, ALL the rules and boundary conditions are abided by.
With the 8 models trained, the features from one test motor will run through the prediction models trained by the remaining 7 motors to obtain 7 predicted health index. This Health index is then mapped to RUL through interpolation or extrapolation.
To gauge the performance of this method, we utilise two error metrics and compare them with the direct method. The direct method maps features directly to the RUL during training. The first error metric is root mean square error which measures the differences between the true RUL and the predicted RUL for every cycle What we observe is that RUL prediction using physics-based machine learning out-performs the direct method for most engines and on average, reduces the root mean square error by 31.5%.
The second error metric used was the mean absolute percentage error which is a measure of prediction accuracy.
Similar to root mean square error, the RUL prediction using physics-based machine learning out-performs the direct method for most engines and on average, reduces the mean absolute percentage error by 33.6%.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent