EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT
Sahar ZAMANI KHANGHAH
zamani.shr@gmail.com(Iran, Islamic Republic of)
https://orcid.org/0009-0009-0485-092X
Keivan MAGHOOLI
Department of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. (Iran, Islamic Republic of)
Abstract
The best emotion recognition system based on physiological signals with a simple operatory should have higher accuracy and fast response speed. This paper aims to develop an emotion recognition system using a novel hybrid system based on Hidden Markov Model and Poincare plot. For this purpose, an electrocardiogram from the MAHNOB-HCI database was used. A novel feature extraction from a hybrid system combining Hidden Markov Model and Poincare plot was presented. The authors extracted time and frequency domain features from heart rate variability, and used two central hybrid systems, the Support Vector Machine/ Hidden Markov Model and the Hidden Markov Model/ Poincare Plot. Finally, the support vector machine was used as a classifier to classify emotions into positive and negative. The proposed method showed a classification accuracy of 95.02 ± 1.97 % overall. Also, the computing time of the method is around 163 milliseconds. The key of this paper is in the use of hybrid machines to improve accuracy without high computation time. This method can be used as a real-time system due to the low computation time and can be developed in many fields, such as medical examination and security systems.
Keywords:
Electrocardiogram, Features extraction, Support vector machine, Emotion classificationReferences
Baghizadeh, M., Maghooli, K., Farokhi, F., & Dabanloo, N. J. (2020). A new emotion detection algorithm using extracted features of the different time-series generated from ST intervals Poincaré map. Biomedical Signal Processing and Control, 59. https://doi.org/10.1016/j.bspc.2020.101902
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
Bong, S. Z., Murugappan, M., & Yaacob, S. (2012, 2012//). Analysis of Electrocardiogram (ECG) Signals for Human Emotional Stress Classification. Trends in Intelligent Robotics, Automation, and Manufacturing, Berlin, Heidelberg.
Bulagang, A. F., Weng, N. G., Mountstephens, J., & Teo, J. (2020). A review of recent approaches for emotion classification using electrocardiography and electrodermography signals. Informatics in Medicine Unlocked, 20. https://doi.org/10.1016/j.imu.2020.100363
Burby, J. W., Tang, Q., & Maulik, R. (2021). Fast neural Poincaré maps for toroidal magnetic fields. Plasma Physics and Controlled Fusion, 63(2), 024001. https://doi.org/10.1088/1361-6587/abcbaa
Ferdinando, H., Seppänen, T., & Alasaarela, E. (2016, 5-7 Oct. 2016). Comparing features from ECG pattern and HRV analysis for emotion recognition system. 2016 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB),
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017). Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Australas Phys Eng Sci Med, 40(2), 277-287. https://doi.org/10.1007/s13246-017-0530-x
Guzik, P., Piskorski, J., Krauze, T., Wykretowicz, A., & Wysocki, H. (2006). Heart rate asymmetry by Poincare plots of RR intervals. Biomed Tech (Berl), 51(4), 272-275. https://doi.org/10.1515/BMT.2006.054
Hoshi, R. A., Pastre, C. M., Vanderlei, L. C., & Godoy, M. F. (2013). Poincare plot indexes of heart rate variability: relationships with other nonlinear variables. Auton Neurosci, 177(2), 271-274. https://doi.org/10.1016/j.autneu.2013.05.004
K, T., Ab. Aziz, N. A., Emerson Raja, J., Hossen, J., & M. Z. H, J. (2022). A Systematic Review on Emotion Recognition System Using Physiological Signals: Data Acquisition and Methodology. Emerging Science Journal, 6(5), 1167-1198. https://doi.org/10.28991/esj-2022-06-05-017
Kim, J., & Andre, E. (2008). Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell, 30(12), 2067-2083. https://doi.org/10.1109/TPAMI.2008.26
Kim, K. H., Bang, S. W., & Kim, S. R. (2004). Emotion recognition system using short-term monitoring of physiological signals. Medical and Biological Engineering and Computing, 42(3), 419-427. https://doi.org/10.1007/BF02344719
Krüger, S., Schafföner, M., Katz, M., Andelic, E., & Wendemuth, A. (2005). Speech recognition with support vector machines in a hybrid system. https://doi.org/10.21437/Interspeech.2005-237
Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A Self-Adaptive Hidden Markov Model for Emotion Classification in Chinese Microblogs. Mathematical Problems in Engineering, 2015, 1-8. https://doi.org/10.1155/2015/987189
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209-237. https://doi.org/10.1080/02699930802204677
Mikuckas, A., Mikuckiene, I., Venckauskas, A., Kazanavicius, E., Lukas, R., & Plauska, I. (2014). Emotion Recognition in Human Computer Interaction Systems. Elektronika ir Elektrotechnika, 20(10). https://doi.org/10.5755/j01.eee.20.10.8878
Moharreri, S., Dabanloo, N. J., & Maghooli, K. (2018). Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybernetics and Biomedical Engineering, 38(4), 794-809. https://doi.org/10.1016/j.bbe.2018.07.001
Park, S., & Kim, K. (2011). Physiological reactivity and facial expression to emotion-inducing films in patients with schizophrenia. Arch Psychiatr Nurs, 25(6), e37-47. https://doi.org/10.1016/j.apnu.2011.08.001
Patlar Akbulut, F., Perros, H. G., & Shahzad, M. (2020). Bimodal affect recognition based on autoregressive hidden Markov models from physiological signals. Comput Methods Programs Biomed, 195, 105571. https://doi.org/10.1016/j.cmpb.2020.105571
Shaffer, F., & Ginsberg, J. P. (2017). An Overview of Heart Rate Variability Metrics and Norms. Front Public Health, 5, 258. https://doi.org/10.3389/fpubh.2017.00258
Soleymani, M., Lichtenauer, J., Pun, T., & Pantic, M. (2012). A Multimodal Database for Affect Recognition and Implicit Tagging. IEEE Transactions on Affective Computing, 3(1), 42-55. https://doi.org/10.1109/t-affc.2011.25
Stadermann, J., & Rigoll, G. (2004). A hybrid SVM/HMM acoustic modeling approach to automatic speech recognition Interspeech 2004,
Wang, B., Liu, D., Gao, X., Luo, Y., & Xi, J. (2022). Three-Dimensional Poincaré Plot Analysis for Heart Rate Variability. Complexity, 2022, 1-9. https://doi.org/10.1155/2022/3880047
Wiem, M., & Lachiri, Z. (2017). Emotion Classification in Arousal Valence Model using MAHNOB-HCI Database. International Journal of Advanced Computer Science and Applications, 8. https://doi.org/10.14569/IJACSA.2017.080344
Zhu, J., Ji, L., & Liu, C. (2019). Heart rate variability monitoring for emotion and disorders of emotion. Physiol Meas, 40(6), 064004. https://doi.org/10.1088/1361-6579/ab1887
ZAMANI KHANGHAH, S., & MAGHOOLI, K. . (2024). EMOTION RECOGNITION FROM HEART RATE VARIABILITY WITH A HYBRID SYSTEM COMBINED HIDDEN MARKOV MODEL AND POINCARE PLOT. Applied Computer Science, 20(1), 106–121. https://doi.org/10.35784/acs-2024-07
Authors
Sahar ZAMANI KHANGHAHzamani.shr@gmail.com
Iran, Islamic Republic of
https://orcid.org/0009-0009-0485-092X
Authors
Keivan MAGHOOLIDepartment of Biomedical Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran. Iran, Islamic Republic of
Statistics
Abstract views: 432PDF downloads: 124
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Similar Articles
- Wulan Dewi, Wiranto Herry Utomo, PLANT CLASSIFICATION BASED ON LEAF EDGES AND LEAF MORPHOLOGICAL VEINS USING WAVELET CONVOLUTIONAL NEURAL NETWORK , Applied Computer Science: Vol. 17 No. 1 (2021)
- Nancy WOODS, Gideon BABATUNDE, A ROBUST ENSEMBLE MODEL FOR SPOKEN LANGUAGE RECOGNITION , Applied Computer Science: Vol. 16 No. 3 (2020)
- Anna MACHROWSKA, Robert KARPIŃSKI, Józef JONAK, Jakub SZABELSKI, NUMERICAL PREDICTION OF THE COMPONENT-RATIO-DEPENDENT COMPRESSIVE STRENGTH OF BONE CEMENT , Applied Computer Science: Vol. 16 No. 3 (2020)
- Błażej CZAJKA, Patryk RÓŻYŁO, Hubert DĘBSKI, STABILITY AND FAILURE OF THIN-WALLED COMPOSITE STRUCTURES WITH A SQUARE CROSS-SECTION , Applied Computer Science: Vol. 18 No. 2 (2022)
- Mohamed ELBAHRI, Nasreddine TALEB, Sid Ahmed El Mehdi ARDJOUN, Chakib Mustapha Anouar ZOUAOUI , FEW-SHOT LEARNING WITH PRE-TRAINED LAYERS INTEGRATION APPLIED TO HAND GESTURE RECOGNITION FOR DISABLED PEOPLE , Applied Computer Science: Vol. 20 No. 2 (2024)
- Anna MACHROWSKA, Robert KARPIŃSKI, Przemysław KRAKOWSKI, Józef JONAK, DIAGNOSTIC FACTORS FOR OPENED AND CLOSED KINEMATIC CHAIN OF VIBROARTHROGRAPHY SIGNALS , Applied Computer Science: Vol. 15 No. 3 (2019)
- Mohanad ABDULHAMID, Njagi KINYUA, SOFTWARE FOR RECOGNITION OF CAR NUMBER PLATE , Applied Computer Science: Vol. 16 No. 1 (2020)
- Rowell HERNANDEZ, Robert ATIENZA, CAREER TRACK PREDICTION USING DEEP LEARNING MODEL BASED ON DISCRETE SERIES OF QUANTITATIVE CLASSIFICATION , Applied Computer Science: Vol. 17 No. 4 (2021)
- Pascal Krutz, Matthias Rehm, Holger Schlegel, Martin Dix, RECOGNITION OF SPORTS EXERCISES USING INERTIAL SENSOR TECHNOLOGY , Applied Computer Science: Vol. 19 No. 1 (2023)
- Gamze Ogcu KAYA, Ali TURKYILMAZ, INTERMITTENT DEMAND FORECASTING USING DATA MINING TECHNIQUES , Applied Computer Science: Vol. 14 No. 2 (2018)
<< < 1 2 3 4 5 6 7 8 9 10 > >>
You may also start an advanced similarity search for this article.
