EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT
Sahar ZAMANI KHANGHAH
zamani.shr@gmail.com(Iran, Islamic Republic of)
https://orcid.org/0009-0009-0485-092X
Keivan MAGHOOLI
Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. (Iran, Islamic Republic of)
Abstract
The best emotion recognition system based on physiological signals with a simple operatory should have higher accuracy and fast response speed. This paper aims to develop an emotion recognition system using a novel hybrid system based on Hidden Markov Model and Poincare plot. For this purpose, an electrocardiogram from the MAHNOB-HCI database was used. A novel feature extraction from a hybrid system combining Hidden Markov Model and Poincare plot was presented. The authors extracted time and frequency domain features from heart rate variability, and used two central hybrid systems, the Support Vector Machine/ Hidden Markov Model and the Hidden Markov Model/ Poincare Plot. Finally, the support vector machine was used as a classifier to classify emotions into positive and negative. The proposed method showed a classification accuracy of 95.02 ± 1.97 % overall. Also, the computing time of the method is around 163 milliseconds. The key of this paper is in the use of hybrid machines to improve accuracy without high computation time. This method can be used as a real-time system due to the low computation time and can be developed in many fields, such as medical examination and security systems.
Keywords:
Electrocardiogram, Features extraction, Support vector machine, Emotion classificationReferences
Baghizadeh, M., Maghooli, K., Farokhi, F., & Dabanloo, N. J. (2020). A new emotion detection algorithm using extracted features of the different time-series generated from ST intervals Poincaré map. Biomedical Signal Processing and Control, 59. https://doi.org/10.1016/j.bspc.2020.101902
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
ZAMANI KHANGHAH, S., & MAGHOOLI, K. . (2024). EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT. Applied Computer Science, 20(1), 106–121. https://doi.org/10.35784/acs-2024-07
Authors
Sahar ZAMANI KHANGHAHzamani.shr@gmail.com
Iran, Islamic Republic of
https://orcid.org/0009-0009-0485-092X
Authors
Keivan MAGHOOLIDepartment of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. Iran, Islamic Republic of
Statistics
Abstract views: 432PDF downloads: 124
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Similar Articles
- Workineh TESEMA, INEFFICIENCY OF DATA MINING ALGORITHMS AND ITS ARCHITECTURE: WITH EMPHASIS TO THE SHORTCOMING OF DATA MINING ALGORITHMS ON THE OUTPUT OF THE RESEARCHES , Applied Computer Science: Vol. 15 No. 3 (2019)
- Arkadiusz GOLA, Łukasz WIECHETEK, MODELLING AND SIMULATION OF PRODUCTION FLOW IN JOB-SHOP PRODUCTION SYSTEM WITH ENTERPRISE DYNAMICS SOFTWARE , Applied Computer Science: Vol. 13 No. 4 (2017)
- Yehor TATARCHENKO, Volodymyr LYFAR, Halyna TATARCHENKO, INFORMATION MODEL OF SYSTEM OF SUPPORT OF DECISION MAKING DURING MANAGEMENT OF IT COMPANIES , Applied Computer Science: Vol. 16 No. 1 (2020)
- Lukas BAUER, Leon STÜTZ, Markus KLEY, BLACK BOX EFFICIENCY MODELLING OF AN ELECTRIC DRIVE UNIT UTILIZING METHODS OF MACHINE LEARNING , Applied Computer Science: Vol. 17 No. 4 (2021)
- Krzysztof Michalczyk, Mariusz Warzecha, Robert Baran, A NEW METHOD FOR GENERATING VIRTUAL MODELS OF NONLINEAR HELICAL SPRINGS BASED ON A RIGOROUS MATHEMATICAL MODEL , Applied Computer Science: Vol. 19 No. 2 (2023)
- Dias Satria, PREDICTING BANKING STOCK PRICES USING RNN, LSTM, AND GRU APPROACH , Applied Computer Science: Vol. 19 No. 1 (2023)
- Lei Liu, Eric B. Blancaflor, Mideth Abisado, A LIGHTWEIGHT MULTI-PERSON POSE ESTIMATION SCHEME BASED ON JETSON NANO , Applied Computer Science: Vol. 19 No. 1 (2023)
- Piotr WITTBRODT, Iwona ŁAPUŃKA, Gulzhan BAYTIKENOVA, Arkadiusz GOLA, Alfiya ZAKIMOVA, IDENTIFICATION OF THE IMPACT OF THE AVAILABILITY FACTOR ON THE EFFICIENCY OF PRODUCTION PROCESSES USING THE AHP AND FUZZY AHP METHODS , Applied Computer Science: Vol. 18 No. 4 (2022)
- Ali Fattah DAKHIL, Weffa Muhammed ALI, Ali Atshan Abdul REDA, PRIORITIZING SOFTWARE CAPABILITIES AND FOCAL POINTS OF MS ACCESS AND EXCEL IN PERSPECTIVE OF DATA MANAGEMENT , Applied Computer Science: Vol. 14 No. 3 (2018)
- Karolina FERYSIUK, Karolina M. WÓJCIAK, THE SPECTROPHOTOMETRIC ANALYSIS OF ANTIOXIDANT PROPERTIES OF SELECTED HERBS IN VISION-PRO™ UV-VIS , Applied Computer Science: Vol. 15 No. 4 (2019)
You may also start an advanced similarity search for this article.
