cpyang/caterpillar_mset

Anomaly detection via Multivariate State Estimation Technique.

What is this?

This is a small numpy/scipy implementation of Multivariate State Estimation technique; in particular of an "online" approach combined with time-delay embeddings to do anomaly detection with scalar time series.

Tools to build a fixed dictionary, implement strategies to split train/detection phases in the absence of ground truth may be added later.

Requirements

Python packages:

• numpy
• scipy
• matplotlib (for visualization)

Getting started

This is not designed to be installed with a package manager (no pip/conda/etc). Simply clone the repository where you wish and use the files mset.py and tde.py as modules which you can import.

The script example.py runs an example of the online anomaly detection on the synthetic time series shown at the top of this readme.

More generally, you may use mset.py as a package. If you have a vector-valued time series Y, where the data is arranged in columns, the simplest call is

import mset
anomalies = mset.online_mset(Y)

Citation

If you would like to credit use of this code with a citation for a paper, please reference (or pull the reference via citation manager) of the following paper:

• Aminian Manuchehr, Andrews-Polymenis Helene, Gupta Jyotsana, Kirby Michael, Kvinge Henry, Ma Xiaofeng, Rosse Patrick, Scoggin Kristin and Threadgill David 2020. Mathematical methods for visualization and anomaly detection in telemetry datasets. Interface Focus.102019008620190086 http://doi.org/10.1098/rsfs.2019.0086

The code implements nonlinear operators suggested by the following papers (though by default uses the operator in the "Wang" paper). Note that you can replace with your own similarity operator if you wish.

• Joshua Thompson, David W. Dreisigmeyer, Terry Jones, Michael Kirby, and Joshua Ladd. 2010. Accurate fault prediction of BlueGene/P RAS logs via geometric reduction. In Proceedings of the 2010 International Conference on Dependable Systems and Networks Workshops (DSN-W) (DSNW '10). IEEE Computer Society, USA, 8–14. DOI: https://doi.org/10.1109/DSNW.2010.5542626
• Wang, K., Thompson, J., Peterson, C., & Kirby, M.J. (2015). Identity maps and their extensions on parameter spaces: Applications to anomaly detection in video. 2015 Science and Information Conference (SAI), 345-351. DOI: https://doi.org/10.1109/SAI.2015.7237167