EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT
Sahar ZAMANI KHANGHAH
zamani.shr@gmail.com(Iran, Islamic Republic of)
https://orcid.org/0009-0009-0485-092X
Keivan MAGHOOLI
Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. (Iran, Islamic Republic of)
Abstract
The best emotion recognition system based on physiological signals with a simple operatory should have higher accuracy and fast response speed. This paper aims to develop an emotion recognition system using a novel hybrid system based on Hidden Markov Model and Poincare plot. For this purpose, an electrocardiogram from the MAHNOB-HCI database was used. A novel feature extraction from a hybrid system combining Hidden Markov Model and Poincare plot was presented. The authors extracted time and frequency domain features from heart rate variability, and used two central hybrid systems, the Support Vector Machine/ Hidden Markov Model and the Hidden Markov Model/ Poincare Plot. Finally, the support vector machine was used as a classifier to classify emotions into positive and negative. The proposed method showed a classification accuracy of 95.02 ± 1.97 % overall. Also, the computing time of the method is around 163 milliseconds. The key of this paper is in the use of hybrid machines to improve accuracy without high computation time. This method can be used as a real-time system due to the low computation time and can be developed in many fields, such as medical examination and security systems.
Keywords:
Electrocardiogram, Features extraction, Support vector machine, Emotion classificationReferences
Baghizadeh, M., Maghooli, K., Farokhi, F., & Dabanloo, N. J. (2020). A new emotion detection algorithm using extracted features of the different time-series generated from ST intervals Poincaré map. Biomedical Signal Processing and Control, 59. https://doi.org/10.1016/j.bspc.2020.101902
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
ZAMANI KHANGHAH, S., & MAGHOOLI, K. . (2024). EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT. Applied Computer Science, 20(1), 106–121. https://doi.org/10.35784/acs-2024-07
Authors
Sahar ZAMANI KHANGHAHzamani.shr@gmail.com
Iran, Islamic Republic of
https://orcid.org/0009-0009-0485-092X
Authors
Keivan MAGHOOLIDepartment of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. Iran, Islamic Republic of
Statistics
Abstract views: 485PDF downloads: 141
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Similar Articles
- Baigo HAMUNA, Sri PUJIYATI, Jonson Lumban GAOL, Totok HESTIRIANOTO, CLASSIFICATION AND PREDICTION OF BENTHIC HABITAT FROM SCIENTIFIC ECHOSOUNDER DATA: APPLICATION OF MACHINE LEARNING ALGORITHMS , Applied Computer Science: Vol. 20 No. 4 (2024)
- Nataliya SHABLIY, Serhii LUPENKO, Nadiia LUTSYK, Oleh YASNIY, Olha MALYSHEVSKA, KEYSTROKE DYNAMICS ANALYSIS USING MACHINE LEARNING METHODS , Applied Computer Science: Vol. 17 No. 4 (2021)
- Anitha Rani PALAKAYALA, Kuppusamy P, A QUALITATIVE AND QUANTITATIVE APPROACH USING MACHINE LEARNING AND NON-MOTOR SYMPTOMS FOR PARKINSON’S DISEASE CLASSIFICATION. A HIERARCHICAL STUDY , Applied Computer Science: Vol. 20 No. 3 (2024)
- Robert KARPIŃSKI, Przemysław KRAKOWSKI, Józef JONAK, Anna MACHROWSKA, Marcin MACIEJEWSKI, COMPARISON OF SELECTED CLASSIFICATION METHODS BASED ON MACHINE LEARNING AS A DIAGNOSTIC TOOL FOR KNEE JOINT CARTILAGE DAMAGE BASED ON GENERATED VIBROACOUSTIC PROCESSES , Applied Computer Science: Vol. 19 No. 4 (2023)
- Mouna TARIK, Ayoub MNIAI, Khalid JEBARI, HYBRID FEATURE SELECTION AND SUPPORT VECTOR MACHINE FRAMEWORK FOR PREDICTING MAINTENANCE FAILURES , Applied Computer Science: Vol. 19 No. 2 (2023)
- Reehana SHAIK, Ibrahim SIDDIQUE , NOVEL MULTI-MODAL OBSTRUCTION MODULE FOR DIABETES MELLITUS CLASSIFICATION USING EXPLAINABLE MACHINE LEARNING , Applied Computer Science: Vol. 20 No. 4 (2024)
- Tilla IZSÁK, László MARÁK, Mihály ORMOS, EVALUATION OF SUPPORT VECTOR MACHINE BASED STOCK PRICE PREDICTION , Applied Computer Science: Vol. 19 No. 3 (2023)
- Islam MOHAMED, Mohamed EL-WAKAD, Khaled ABBAS, Mohamed ABOAMER, Nader A. Rahman MOHAMED, PUPIL DIAMETER AND MACHINE LEARNING FOR DEPRESSION DETECTION: A COMPARATIVE STUDY WITH DEEP LEARNING MODELS , Applied Computer Science: Vol. 20 No. 4 (2024)
- Nawazish NAVEED, Hayan T. MADHLOOM, Mohd Shahid HUSAIN, BREAST CANCER DIAGNOSIS USING WRAPPER-BASED FEATURE SELECTION AND ARTIFICIAL NEURAL NETWORK , Applied Computer Science: Vol. 17 No. 3 (2021)
- Jerzy JÓZWIK, Magdalena ZAWADA-MICHAŁOWSKA, Monika KULISZ, Paweł TOMIŁO, Marcin BARSZCZ, Paweł PIEŚKO, Michał LELEŃ, Kamil CYBUL, MODELING THE OPTIMAL MEASUREMENT TIME WITH A PROBE ON THE MACHINE TOOL USING MACHINE LEARNING METHODS , Applied Computer Science: Vol. 20 No. 2 (2024)
You may also start an advanced similarity search for this article.
