ghksdl6025/online_when2update

Exploring the Performance Dynamics of Online Predictive Process Monitoring using Log Features

Abstract

Predictive process monitoring is essential for forecasting on-going process outcomes. However, in practical online settings fluctuations of prediction performance are problematic. While existing research addresses performance drifts or stability measures, understanding the underlying causes of these fluctuations remains limited. This study aims to fill this knowledge gap by investigating factors contributing to performance variability in prediction models for online process outcome predictions. Through an analysis of log features and their correlation with model performance over time, insights are provided to inform strategies for mitigating performance drops and enhancing model reliability. Our results show a high correlation between log features and the model performance in many cases. Retraining the prediction models based on these insights shows accordingly shows improvement in both average performance and stability. Understanding these factors enables the development of resilient models and tailored retraining strategies, avoiding unnecessary retraining, energy consumption, and costs.

Method

Log features

1. Variant coverage This feature measures the appearance of newly observed variants in the test data, which were not included during model training.

2. Label distribution This log feature measures the appearance of newly observed categorical event-level attributes in the test data.

3. Event level attributes coverage This log feature measures the appearance of newly observed categorical event-level attributes in the test data.

Results