Machine Learning models for the Cognitive Stress Detection Using Heart Rate Variability Signals
:
https://doi.org/10.9744/jti.24.2.83-94
Keywords:
cognitive, human factors, heart rate variability, work performance, machine learningAbstract
Cognitive domains play a critical role in daily functioning. The prediction of cognitive stress state is important to better monitor work performance. This study aims to explore machine learning models to detect cognitive load or state using heart rate variability (HRV) signals. HRV data were recorded from thirty subjects during rest, two cognitive tasks (d2 Attention and Featuring Switcher task), and recovery. Seven HRV indexes from both time and frequency domains, extracted from raw R-R intervals, were used to identify whether subjects performed cognitive tasks or not. Five classifier models: linear support vector machine (LSVM), kernel SVM radial basis function, k-nearest neighbor (KNN), and random forest (RF), were trained and evaluated using a leave-one-out cross-validation approach. The accuracies and F1-score range from 0.54 to 0.62, with LSVM, showing the best. These acceptable performances indicate the machine learning approach could be used to further distinguish between rest and cognitive state. With the ubiquity of non-invasive and low-cost wearable devices, this finding offers insight to be incorporated into personal work performance monitoring in the digital age.
References
Weintraub, S., Dikmen, S.S., Heaton, R.K., Tulsky, D.S., Zelazo, P.D., and Bauer, P.J., Cognition Assessment Using the NIH Toolbox, Neurology, 80(11), 2013, pp. S54–S64. doi: 10.1212/WNL.0b013e3182872ded.
Shields, G.S., Sazma, M.A., and Yonelinas A.P., The Effects of Acute Stress on Core Executive Functions: A Meta-Analysis and Comparison with Cortisol, Neuroscience Biobehavioral Review, 68, 2016, pp.651-668. doi: 10.1016/j.neubiorev.2016.06.038.
Tervonen, J., Pettersson, K., and Mäntyjärvi, J., Ultra-short Window Length and Feature Importance Analysis for Cognitive Load Detection from Wearable Sensors, Electronics (Switzerland)., 10(5), 2021, pp.1-19. doi: 10.3390/electronics10050613.
Sharma, K., Niforatos, E., Giannakos, M., and Kostakos V., Assessing Cognitive Performance Using Physiological and Facial Features, Proceedings of ACM Interactive, Mobile, Wearable Ubiquitous Technology, 4(3), 2020, doi: 10.1145/3411811.
Thielmann, B, Pohl, R., and Böckelmann, I., Heart Rate Variability as a Strain Indicator for Psychological Stress for Emergency Physicians During Work and Alert Intervention: A Systematic Review, Journal of Occupational Medicine Toxicology, 16(1), 2021, pp. 1-9. doi: 10.1186/s12995-021-00313-3.
Castaldo, R., Melillo, R., Bracale, U., Caserta, M. Triassi, M., and Pecchia, L., Acute Mental Stress Assessment via Short Term HRV Analysis in Healthy Adults: A Systematic Review with Meta-Analysis, Biomedical and Signal Processing Control, 18(April), 2015, pp. 370–377. doi: 10.1016/j.bspc.2015.02.012.
Lee, K. F. A., Gan, W.S, and Christopoulos, G., Biomarker-Informed Machine Learning Model of Cognitive Fatigue from a Heart Rate Response Perspective, Sensors, 21(11), 2021, pp. 1–16. doi: 10.3390/s21113843.
Shaffer, F., and Ginsberg, J.P., An Overview of Heart Rate Variability Metrics and Norms, Frontiers in Public Health., 5(258), 2017, doi: 10.3389/fpubh.2017.00258.
Thayer, J. F., Hansen, A. L., Saus-Rose, E., and Johnsen, B. H. ., Heart Rate Variability, Prefrontal Neural Function, and Cognitive Performance: The Neurovisceral Integration Perspective on Self-Regulation, Adaptation, and Health, Annals of Behavioral Medicine, 37(2), 2009, pp. 141–153. doi: 10.1007/s12160-009-9101-z.
Hoffman, T., What Is Heart Rate Variability (HRV) & Why Does It Matter?, Firstbeat, 2022, retrieved from https://www.firstbeat.com/en/blog/what-is-heart-rate-variability-hrv/ on 17 September 2022
Laborde, S., Mosley, E., and Thayer, J. F., Heart Rate Variability and Cardiac Vagal Tone in Psychophysiological Research - Recommendations for Experiment Planning, Data Analysis, and Data Reporting, Frontiers in Psychology, 8(213), 2017, doi: 10.3389/fpsyg.2017.00213.
Kim, H. G. , Cheon, E. J., Bai, D. S., Lee, Y. H., and Koo, B. H., Stress and Heart Rate Variability: A Meta-Analysis and Review of the Literature, Psychiatry Investigation, 15(3), 2018, pp. 235–245. doi: 10.30773/pi.2017.08.17.
Massaro, S., and Pecchia, L., Heart Rate Variability (HRV) Analysis: A Methodology for Organizational Neuroscience, Organizational Research Methods, 22(1), 2019, pp. 354-393
Yarkoni, T., and Westfall J., Choosing Prediction Over Explanation in Psychology: Lessons from Machine Learning, Perspective in Psychology Science., 12(6), 2017, pp. 1100–1122. doi: 10.1177/1745691617693393.
Giannakakis, J., Marias, K., and Tsiknakis, M., A Stress Recognition System using HRV Parameters and Machine Learning Techniques, Proceedings of 2019 8th International Conference on Affective Computing and Intelligent Interaction, 2019, pp. 269–272. doi: 10.1109/ACIIW.2019.8925142.
Chen, C., Li, C., Tsai, C.-W., and Deng, X., Evaluation of Mental Stress and Heart Rate Variability Derived from Wrist-Based Photoplethysmography, Proceedings of 2019 IEEE Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS), 2019, pp. 65–68. doi: 10.1109/ECBIOS.2019.8807835.
Lee, S., Hwang, H.B., Park, S., Kim, S., Ha, J.J., Jang, Y., Hwang, S., Park, H., Lee, J., and Kim, I.Y., Mental Stress Assessment Using Ultra Short Term HRV Analysis Based on Non-Linear Method, Biosensors (Basel), 12(7), 2022, doi: 10.3390/bios12070465.
Stewart, C. L., Folarin, A., and Dobson, R., Personalized Acute Stress Classification from Physiological Signals with Neural Processes, 2020, retrieved from http://arxiv.org/abs/2002.04176 on 23 June 2022
Ahmad, Z., and Khan, N. M., Multi-level Stress Assessment Using Multi-Domain Fusion of ECG Signal, Proceedings of Annual International Conference IEEE Engineering, in Medicine and Biological Society, 2020, pp. 4518–4521. doi: 10.1109/EMBC44109.2020.9176590.
Betti, S., Lova, R.M, Rovini, E., Acerbi, G., Santarelli, L., Cabiati, M., Ry, S.D., and Cavallo, F., Evaluation of an Integrated System of Wearable Physiological Sensors for Stress Monitoring in Working Environments by Using Biological Markers, IEEE Transaction on Biomedical Engineering, 65(8), 2018, pp. 1748–1758. doi: 10.1109/TBME.2017.2764507.
Sanchez, W., Martinez, A., Hernandez, Y., Estrada, H., and Gonzalez-Mendoza, M., A Predictive Model for Stress Recognition in Desk Jobs, Journal of Ambient and Intelligence of Humanized Computing, 2018, doi: 10.1007/s12652-018-1149-9.
Can, Y. S., Arnrich, B., and Ersoy, C., Stress Detection in Daily Life Scenarios Using Smart Phones and Wearable Sensors: A Survey, Journal of Biomedical Informatics, 92(August), 2019, 103139. doi: 10.1016/j.jbi.2019.103139.
Hörmann, T., Hesse, M., Christ, P., Adams, M., Menßen, C., and Rückert, U., Fine-Grained Prediction of Cognitive Workload in a Modern Working Environment by Utilizing Short-Term Physiological Parameters, Proceedings of BIOSIGNALS 2016 - 9th International Conference on Bio-Inspired System and Signal Processing, 2016, pp. 42–51, 2016, doi: 10.5220/0005665000420051.
Forte, G., Favieri, F., and Casagrande, M., Heart Rate Variability and Cognitive Function: A Systematic Review, Frontiers in Neuroscience, 13(July), 2019, doi: 10.3389/fnins.2019.00710.
Tsunoda, K., Chiba, A., Yoshida, K., Tomoki, W., and Mizuno, O., Predicting Changes in Cognitive Performance Using Heart Rate Variability, IEICE Transacation on Information System. E100.D(10), 2017, pp. 2411–2419. doi: 10.1587/transinf.2016OFP0002.
Wang, C., and Guo, J., A Data-Driven Framework for Learners’ Cognitive Load Detection Using ECG-PPG Physiological Feature Fusion and XGBoost Classification, Procedia Computer Science, 147, 2019, pp. 338–348. doi: 10.1016/j.procs.2019.01.234.
Castaldo, R., Montesinos, L., Melillo, P., James, C., and Pecchia, L., Ultra-Short Term HRV Features as Surrogates of Short Term HRV: A Case Study on Mental Stress Detection in Real Life, BMC Medical Informatics and Decision Making, 19(1), 2019, pp. 1–13. doi: 10.1186/s12911-019-0742-y.
Posada-Quintero, H. F., and Bolkhovsky, J. B., Machine Learning Models for the Identification of Cognitive Tasks Using Autonomic Reactions from Heart Rate Variability and Electrodermal Activity, Behavioral Science (Basel)., 9(4), 2019, doi: 10.3390/bs9040045.
Huang, S., Li, J., Zhang, P., and Zhang, W., Detection of Mental Fatigue State with Wearable ECG Devices, International Journal of Medical Informatics, 19(August), 2018, pp. 39–46. doi: 10.1016/j.ijmedinf.2018.08.010.
Alberdi, A., Aztiria, A., and Basarab, A., Towards an Automatic Early Stress Recognition System for Office Environments based on Multimodal Measurements: A Review, Journal of Biomedical Informatics, 59(February), 2016, pp. 49–75. doi: 10.1016/j.jbi.2015.11.007.
Castaldo, R., Montesinos, L., Wan, T. S., Serban, A., Massaro, S., and Pecchia, L., Heart Rate Variability Analysis and Performance during a Repeated Mental Workload Task, Proceedings of IFMBE, 65, 2018, pp. 69–72.
Lambusch, F., Weigelt, O., Fellmann, M., Hein, S., and Poppe, M., Personal Productivity Management in the Digital Age: Measures from Research and Use of Conventional Tools, Proceedings of 15th International Conference on Business and Information System WIRTSCHAFTSINFORMATIK, 2020, doi: 10.30844/wi_2020_f5.
McDuff, D., Gontarek, S., and Picard, R., Remote Measurement of Cognitive Stress via Heart Rate Variability, Proceedings of 2014 36th Annual International Conference of the Engineering in Medicine and Biology Society, 2014, pp. 2957–2960. doi: 10.1109/EMBC.2014.6944243.
Brickenkamp, R., Test D2, Attentional Performance Test, Hogrefe, Göttingen, Germany, 1994.
Steinborn, M. B., Langner, R., Flehmig, H. C., and Huestegge, L., Methodology of Performance Scoring in the D2 Sustained-Attention Test: Cumulative-Reliability Functions and Practical guidelines, Psychological Assessment, 30(3), 2018, pp. 339–357. doi: 10.1037/pas0000482.
Anderson, K., Deane, K., Lindley, D., Loucks, B., and Veach, E., The Effects of Time of Day and Practice on Cognitive Abilities: The PEBL Tower of London, Trail-Making, and Switcher Tasks, 2012, retrieved from http://sites.google.com/site/pebltechnicalreports/home/2012/pebl-technical-report-2012-04 on 12 June 2022
Mueller, S. T., and Piper, B. J., The Psychology Experiment Building Language (PEBL) and PEBL Test Battery, Journal of Neuroscience Methods, 222(January), 2014, pp. 250–259. doi: 10.1016/j.jneumeth.2013.10.024.
Hinde, K., White, G., and Armstrong, N., Wearable Devices Suitable for Monitoring Twenty-Four Hour Heart Rate Variability in Military Populations, Sensors (Switzerland), 21(4), 2021, doi: 10.3390/s21041061.
Perrotta, A.S., Jeklin, A. T., Hives, B.A., Meanwell, L.E. , and Warburton, D.E.R., Validity of the Elite HRV Smartphone Application for Examining Heart Rate Variability in a Field-Based Setting, Journal of Strength and Conditions Research, 31(8), 2017, pp. 2296–2302. doi: 10.1519/JSC.0000000000001841.
Tarvainen MP., JA, L., J-P, N., and Ranta-Aho, P., Kubios HRV Software User’s Guide, 2021, retrieved from https://www.kubios.com/downloads/Kubios_HRV_Users_Guide.pdf. on 5 May 2022
Ishaque, S., Khan, N., and Krishnan, S., Trends in Heart-Rate Variability Signal Analysis, Frontiers in Digital Health, 3(February), 2021, pp. 1–18. doi: 10.3389/fdgth.2021.639444.
Borisov, V., Kasneci, E., and Kasneci, G., Robust Cognitive Load Detection from Wrist-Band Sensors, Computer and Human Behavioral Reports, 4(June), 2021, doi: 10.1016/j.chbr.2021.100116.
Hastie, T., Tibshirani, R., and Friedman, J., The Elements of Statistical Learning. New York, NY: Springer New York, 2009.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., and Thirion, B., Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, 12, 2011, pp. 2825–2830.
Schmidt, P., Reiss, A., Dürichen R., and Van Laerhoven, K., Wearable-Based Affect Recognition—a Review, Sensors (Switzerland), 19(19), 2019, pp. 1–42. doi: 10.3390/s19194079.
Raschka, S., and Mirjalili, V., Python Machine Learning - Second Edition: Machine Learning and Deep Learning with Python, Scikit-Learn, and TensorFlow. Packt Publishing, 2020.
Pham, T., Lau, Z.J., Chen, S.H.A., and Makowski, D., Heart Rate Variability in Psychology: A Review of HRV Indices and an Analysis Tutorial, Sensors, 21(12) 2021, pp. 1–20. doi: 10.3390/s21123998.
Rodrigues, S., Paiva, J.S., Dias, D., Aleixo, M., Filipe, R. M., and Cunha, J. P. S., Cognitive Impact and Psychophysiological Effects of Stress Using a Biomonitoring Platform, International Journal of Research Publich Health, 15(6), 2018, doi: 10.3390/ijerph15061080.
Smith, T. W., Deits-Lebehn, C., Williams, P. G. , Baucom, B. R. W., and Uchino, B. N., Toward a Social Psychophysiology of Vagally Mediated Heart Rate Variability: Concepts and Methods in Self-Regulation, Emotion, and Interpersonal Processes, Social and Personality Compass, 14(3), 2020, pp. 1–24. doi: 10.1111/spc3.12516.
Al-Libawy, H., Al-Ataby, A., Al-Nuaimy, W., and Al-Taee, M. A., HRV-Based Operator Fatigue Analysis and Classification Using Wearable Sensors, Proceedings of 13th International Multi-Conference on Systems, Signals, and Devices, 2016, pp. 268–273. doi: 10.1109/SSD.2016.7473750.
He, J., Malinovic, A., and Jiang N., Fast Detection of Acute Cognitive Stress Measurement Via Heart Rate Variability, Proceedings of the International of IEEE/EMBS Conference on Neural Engineering, 2019, pp. 445–448. doi: 10.1109/NER.2019.8716939.
Géron, A., Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, 2nd ed. Sebastopol, California: O’Reilly Media, Inc, 2019.
Gjoreski, M., Kolenik, T., Knez, T., Luštrek, M., Gams, M., Gjoreski, H., and Pejovic, V., Datasets for Cognitive Load Inference Using Wearable Sensors and Psychological Traits, Applied Science, 10(11), 2020, doi: 10.3390/app10113843.
Gjoreski, M., Mahesh, B., Kolenik, T., Uwe-Garbas, J., Seuss, D., and Gjoreski, H., Cognitive Load Monitoring with Wearables-Lessons Learned from a Machine Learning Challenge,” IEEE Access, 9, 2021, pp. 103325–103336. doi: 10.1109/ACCESS.2021.3093216.
Gjoreski, M., Luštrek, M., Gams, M., and Gjoreski, H., Monitoring Stress with a Wrist Device Using Context, Journal of Biomedical Informatics, 73, 2017, pp. 159–170. doi: 10.1016/j.jbi.2017.08.006.
Milstein, N., and Gordon, I., Validating Measures of Electrodermal Activity and Heart Rate Variability Derived from the Empatica E4 Utilized in Research Settings that Involve Interactive Dyadic States, Frontiers in Behavioral Neuroscience, 14(August), 2020, doi: 10.3389/fnbeh.2020.00148
Downloads
Published
How to Cite
Issue
Section
License
Articles published in the Jurnal Teknik Industri: Jurnal Keilmuan dan Aplikasi Teknik Industri will be Open-Access articles distributed under the terms and conditions of the Creative Commons Attribution License (CC BY).
This work is licensed under a Creative Commons Attribution License (CC BY).
