walkabillylab/device_validity_time

Study of the validity of wearable devices over time.

Changes in Step Count Accuracy of Wearable Devices with the Continuous Development of New Technology

Authors

1. Daniel Fuller, Department of Community Health and Epidemiology, University of Saskatchewan.
2. Ryan Ellis, School of Human Kinetics and Recreation, Memorial University of Newfoundland.
3. Kevin Mongeon, School of Human Kinetics, University of Ottawa, Canada.

Abstract

Commercial wearable devices have become an important tool in the physical activity routines of many people. An important factor in the usefulness of these devices is their criterion validity to measure various physical activity parameters compared to a gold standard. The purpose of this study is to examine whether criterion validity of commercial wearable devices has changed over time with the release of new devices. A cross-sectional study was conducted using publicly available data from a systematic review examining the criterion validity of commercial wearable devices. The systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We included studies with valid data for the step count mean absolute percent error (MAPE), device market availability, device brand and type, and study year. Regression analyses were used to examine the association between device MAPE and the year the device was available on the market. A total of 944 MAPE values were included in the analysis. Devices were released between 2008 and 2016. Results show that for each year there was an improvement in MAPE of -1.5 percent (CI: -2.5 to -0.5). Sensitivity analysis showed while most brands validity improved over time, Fitbit validity did not change. The results suggest that despite many iterations of new commercial wearables being released there has only been small improvements in criterion of these devices over time.

Data and Code

• Data are available here: https://doi.org/10.7910/DVN/O7GQIM
• Raw RMarkdown is here: https://github.com/walkabillylab/device_validity_time/blob/main/device_validity_time.Rmd
• Clean results are here: https://github.com/walkabillylab/device_validity_time/blob/main/device_validity_time.md