James W. Navalta1, Dustin W. Davis1, Bryson Carrier1, Elias M. Malek1, Nicole Vargas1, Jorge Perdomo Rodriguez1, Bianca Weyers1, Katherine Carlos1, Myranda Peck1
1University of Nevada, Department of Kinesiology and Nutrition Sciences, Las Vegas, Nevada, USA
Validity and Reliability of Wearable Devices during Self-Paced Walking, Jogging and Overground Skipping
Sport Mont 2023, 21(3), 23-29 | DOI: 10.26773/smj.231004
Abstract
Wearable technology can track unusual exercise, providing data for improving fitness. The aim of the study was to determine validity and reliability during walking, jogging, and skipping. Eighteen volunteers completed 5 min self-paced activities interspersed with 5 min rest. Variables and devices were step count (Garmin Instinct), estimated energy expenditure (Garmin Instinct, Polar Vantage M2), and heart rate (Garmin Instinct, Polar Vantage M2, Polar OH1, Polar Verity Sense). Validity measures were mean absolute percent error (MAPE) and Lin’s Concordance (CCC), and reliability were coefficient of variation (CV), and intraclass correlation (ICC). Thresholds were MAPE ≤5%, CCC≥0.90, CV≤10%, ICC≥0.70. Garmin Instinct step count during skipping was not considered valid (MAPE=90.2%, CCC=0.008) or reliable (CV=6%, ICC CI=0.4). Energy expenditure during skipping was not valid or reliable in the Garmin Instinct (MAPE=28%, CCC=0.27; CV=19%, ICC=0.61) or the Polar Vantage M2 (MAPE=19%, CCC=0.57; CV=13%). While the Polar Vantage M2 was reliable for estimated energy expenditure during walking and jogging activities, wrist-worn devices (Garmin Instinct, Polar Vantage M2) were neither valid nor reliable in returning estimated energy expenditure during overground skipping. From a wider perspective, wearable device algorithms for estimating energy expenditure should continue to be refined until they return the same level of accuracy as what is currently observed for heart rate, and to a lesser extent step count. Skipping may be an excellent unusual activity for testing wearable devices.
Keywords
wearable activity tracker, unusual exercise, accuracy, consistency
View full article
(PDF – 114KB)
References
Bodell, N., Carrier, B., Gil, D., Fullmer, W., Cruz, K., Aguilar, C. D., . . . Manning, J. W. (2021). Validity of Average Heart Rate and Energy Expenditure in Polar OH1 and Verity Sense While Self-Paced Walking. Paper presented at the International Journal of Exercise Science: Conference Proceedings.
Bunn, J. A., Navalta, J. W., Fountaine, C. J., & Reece, J. D. (2018). Current State of Commercial Wearable Technology in Physical Activity Monitoring 2015-2017. International Journal of Exercise Science, 11(7), 503-515. Retrieved from https://www.ncbi.nlm.nih.gov/pubmed/29541338
Cadmus-Bertram, L. A., Marcus, B. H., Patterson, R. E., Parker, B. A., & Morey, B. L. (2015). Randomized Trial of a Fitbit-Based Physical Activity Intervention for Women. American Journal of Preventative Medicine, 49(3), 414-418. https://doi.org/10.1016/j.amepre.2015.01.020
Cao, M., Xie, T., & Chen, Z. (2019). Wearable sensors and equipment in VR games: a review. Transactions on Edutainment, XV, 3-12.
Carrier, B., Barrios, B., Jolley, B. D., & Navalta, J. W. (2020). Validity and Reliability of Physiological Data in Applied Settings Measured by Wearable Technology: A Rapid Systematic Review. Technologies, 8(4). https://doi.org/ARTN 70https://doi.org/10.3390/technologies8040070
Duking, P., Giessing, L., Frenkel, M. O., Koehler, K., Holmberg, H. C., & Sperlich, B. (2020). Wrist-Worn Wearables for Monitoring Heart Rate and Energy Expenditure While Sitting or Performing Light-to-Vigorous Physical Activity: Validation Study. JMIR Mhealth Uhealth, 8(5), e16716. https://doi.org/10.2196/16716
Gilgen-Ammann, R., Schweizer, T., & Wyss, T. (2019). RR interval signal quality of a heart rate monitor and an ECG Holter at rest and during exercise. European Journal of Applied Physiology, 119(7), 1525-1532. https://doi.org/10.1007/s00421-019-04142-5
Gottschall, J. S., & Kram, R. (2005). Energy cost and muscular activity required for leg swing during walking. Journal of Applied Physiology, 99(1), 23-30.
Hajj-Boutros, G., Landry-Duval, M. A., Comtois, A. S., Gouspillou, G., & Karelis, A. D. (2023). Wrist-worn devices for the measurement of heart rate and energy expenditure: A validation study for the Apple Watch 6, Polar Vantage V and Fitbit Sense. European Journal of Sport Science, 23(2), 165-177. https://doi.org/10.1080/17461391.2021.2023656
Harrell, J. S., McMurray, R. G., Baggett, C. D., Pennell, M. L., Pearce, P. F., & Bangdiwala, S. I. (2005). Energy Costs of Physical Activities in Children and Adolescents. Medicine & Science in Sports & Exercise, 37(2), 329-336. https://doi.org/10.1249/01.Mss.0000153115.33762.3f
Hermand, E., Cassirame, J., Ennequin, G., & Hue, O. (2019). Validation of a Photoplethysmographic Heart Rate Monitor: Polar OH1. International Journal of Sports Medicine, 40(7), 462-467. https://doi.org/10.1055/a-0875-4033
Hettiarachchi, I. T., Hanoun, S., Nahavandi, D., & Nahavandi, S. (2019). Validation of Polar OH1 optical heart rate sensor for moderate and high intensity physical activities. PLoS One, 14(5), e0217288. https://doi.org/10.1371/journal.pone.0217288
Jakicic, J. M., Davis, K. K., Rogers, R. J., King, W. C., Marcus, M. D., Helsel, D., . . . Belle, S. H. (2016). Effect of Wearable Technology Combined With a Lifestyle Intervention on Long-term Weight Loss: The IDEA Randomized Clinical Trial. JAMA, 316(11), 1161-1171. https://doi.org/10.1001/jama.2016.12858
Liu, L., Wang, H. H., Qiu, S., Zhang, Y. C., & Hao, Z. D. (2021). Paddle Stroke Analysis for Kayakers Using Wearable Technologies. Sensors (Basel), 21(3). https://doi.org/10.3390/s21030914
Montes, J., & Navalta, J. W. (2019). Reliability of the Polar T31 Uncoded Heart Rate Monitor in Free Motion and Treadmill Activities. International Journal of Exercise Science, 12(4), 69-76.
Montes, J., Tandy, R. D., Young, J. C., Lee, S.-P., & Navalta, J. W. (2022). Is Body Composition or Body Mass Index Associated with the Step Count Accuracy of a Wearable Technology Device? Topics in Exercise Science and Kinesiology, 3(1), 5.
Muggeridge, D. J., Hickson, K., Davies, A. V., Giggins, O. M., Megson, I. L., Gorely, T., & Crabtree, D. R. (2021). Measurement of Heart Rate Using the Polar OH1 and Fitbit Charge 3 Wearable Devices in Healthy Adults During Light, Moderate, Vigorous, and Sprint-Based Exercise: Validation Study. JMIR Mhealth Uhealth, 9(3), e25313. https://doi.org/10.2196/25313
Murakami, H., Kawakami, R., Nakae, S., Yamada, Y., Nakata, Y., Ohkawara, K., . . . Miyachi, M. (2019). Accuracy of 12 Wearable Devices for Estimating Physical Activity Energy Expenditure Using a Metabolic Chamber and the Doubly Labeled Water Method: Validation Study. JMIR Mhealth Uhealth, 7(8), e13938. https://doi.org/10.2196/13938
Navalta, J. W., & Bunn, J. A. (2023). Editorial: Continued opportunities in wearable technologies and physiological assessment. Frontiers in Digital Health, 5, 1166088. https://doi.org/10.3389/fdgth.2023.1166088
Navalta, J. W., Davis, D. W., Carrier, B., Sertic, J. V., & Cater, P. (2021). Teaching Applied Exercise Physiology Using a Prototype Energy Expenditure Measurement Device. Interdisciplinary Journal of Problem-based Learning, 15(2), n2.
Navalta, J. W., Montes, J., Bodell, N. G., Aguilar, C. D., Radzak, K., Manning, J. W., & DeBeliso, M. (2019). Reliability of Trail Walking and Running Tasks Using the Stryd Power Meter. International Journal of Sports Medicine, 40(8), 498-502. https://doi.org/10.1055/a-0875-4068
Navalta, J. W., Montes, J., Bodell, N. G., Salatto, R. W., Manning, J. W., & DeBeliso, M. (2020). Concurrent heart rate validity of wearable technology devices during trail running. PLoS One, 15(8), e0238569. https://doi.org/10.1371/journal.pone.0238569
Olstad, B. H., & Zinner, C. (2020). Validation of the Polar OH1 and M600 optical heart rate sensors during front crawl swim training. PLoS One, 15(4), e0231522. https://doi.org/10.1371/journal.pone.0231522
Pinedo-Jauregi, A., Garcia-Tabar, I., Carrier, B., Navalta, J. W., & Camara, J. (2022). Reliability and validity of the Stryd Power Meter during different walking conditions. Gait Posture, 92, 277-283. https://doi.org/10.1016/j.gaitpost.2021.11.041
Reddy, R. K., Pooni, R., Zaharieva, D. P., Senf, B., El Youssef, J., Dassau, E., . . . Jacobs, P. G. (2018). Accuracy of Wrist-Worn Activity Monitors During Common Daily Physical Activities and Types of Structured Exercise: Evaluation Study. JMIR Mhealth Uhealth, 6(12), e10338. https://doi.org/10.2196/10338
Speer, K. E., Semple, S., Naumovski, N., & McKune, A. J. (2020). Measuring Heart Rate Variability Using Commercially Available Devices in Healthy Children: A Validity and Reliability Study. European Journal of Investigation in Health, Psychology and Education, 10(1), 390-404. https://doi.org/10.3390/ejihpe10010029
Stančin, S., & Tomažič, S. (2022). Recognizing Solo Jazz Dance Moves Using a Single Leg-Attached Inertial Wearable Device. Sensors, 22(7), 2446. Retrieved from https://www.mdpi.com/1424-8220/22/7/2446
Tanner, E., Manning, J. W., Taylor, J., Montes, J., McCune, D., Koschel, T., . . . Navalta, J. W. (2016). Validation of Hexoskin Biometric Shirt to Cosmed k4b2 Metabolic Unit in Adults During Trail Running. Medicine and Science in Sports and Exercise, 48(5), 461-461. https://doi.org/10.1249/01.mss.0000486387.43740.5e
The jamovi project. (2023). Jamovi (Version 2.3) [Computer Software]. Retrieved from https://www.jamovi.org
Umek, A., & Kos, A. (2018). Wearable sensors and smart equipment for feedback in watersports. Procedia Computer Science, 129, 496-502.
Verdel, N., Drobnič, M., Maslik, J., Hellman, O., Gumiero, A., Dellatorre, L., . . . Supej, M. (2022). The Validity of Measuring Skipping Cadence with a Novel Wearable Sensor - SINTEC Smart Patch. ISBS Proceedings Archive, 40(1), 728.
Yen, H. Y., & Chiu, H. L. (2019). The effectiveness of wearable technologies as physical activity interventions in weight control: A systematic review and meta-analysis of randomized controlled trials. Obesity Reviews, 20(10), 1485-1493. https://doi.org/10.1111/obr.12909
Yongmao, Z., & Yuxin, L. (2023). Application of wearable devices based on deep learning algorithm in rope skipping data monitoring. Soft Computing, 1-11.