The automatic recognition of pain intensity from facial expression has great potential to enhance the objective and non-invasive assessment of pain. In this study, we explore supervised learning methods based on deep learning for the intensity classification of facial pain in 16 different levels. Three CNN architectures (EfficientNetB0, MobileNetV2, Small CNN) were tested to compare pretrained and task specific approaches to fine-grained pain classification. After suitable face image preprocessing and data augmentation, the models were trained and tested on facial images. The following metrics were used to evaluate the performance: accuracy, balanced accuracy, precision, recall, F1-score, specificity, Cohen’s kappa, Matthews’s correlation coefficient and log loss. EfficientNetB0 had the highest accuracy (98.00%), balanced accuracy (97.61%), precision (97.88%), recall (97.61%) and F1 score (97.74%). The MobileNetV2 reached the accuracy of 95.00% and F1 score of 94.61% while the Small CNN reached the accuracy of 88.00% and F1 score of 87.37%. The efficientNetB0 also had the smallest log loss with 0.0683. The results show that deep learning is a viable approach to fine-grained pain-intensity classification on faces, and that state-of-the-art architectures can be used to discriminate between different pain-levels. External validation and access to more subject diversity and temporal facial information should be investigated further to evaluate robustness in practical and clinical settings.
References
Acevedo, D., Negri, P., Buemi, M. E., Fernandez, F. G., & Mejail, M. (2017). A simple geometric-based descriptor for facial expression recognition. Proceedings of the 12th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2017), 802–808. https://doi.org/10.1109/FG.2017.101
Agha, M. A. M., & Aziz, M. H. (2023). Binary Detection Using for Features Extraction. Tianjin Daxue Xuebao (Ziran Kexue Yu Gongcheng Jishu Ban)/Journal of Tianjin University Science and Technology, (January), 197–208. https://doi.org/10.17605/OSF.IO/BN2DQ
Al-Neama, M. W., Abdulrahman, E. H., & Ali, S. M. (2025). A parallel algorithm for facial expression recognition for student engagement monitoring in classrooms. Mathematical Modelling of Engineering Problems, 12(3).
Al-Neama, M. W., Alshiha, A. A., & Saeed, M. G. (2023). A parallel algorithm of multiple face detection on multi-core system. Indonesian Journal of Electrical Engineering and Computer Science, 29(2), 1166–1173. https://doi.org/10.11591/ijeecs.v29.i2.pp1166-1173
Alshiha, A., Al-Neama, M., & Qubaa, A. (2023). Parallel hybrid algorithm for face recognition using multi-linear methods. International Journal of Electrical and Electronics Research, 11(4), 1013–1021.
Alshiha, A. A., Al-Neama, M. W., & Qubaa, A. R. (2023). Biometric face recognition method using graphics processing unit system. Indonesian Journal of Electrical Engineering and Computer Science, 30(1), 183–191.
Andersen, P. H., et al. (2021). Towards machine recognition of facial expressions of pain in horses. Animals, 11(6), 1643. https://doi.org/10.3390/ani11061643
Bargshady, G. (2020). Enhanced deep learning predictive modelling approaches for pain intensity recognition from facial expression video images (Master’s thesis). Southern Queensland University.
Baumeister, R., & Vohs, K. (2012). Facial expression of emotion. In Encyclopedia of Social Psychology. https://doi.org/10.4135/9781412956253.n209
Ekman, P., & Rosenberg, E. L. (2012). What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195179644.001.0001
El Morabit, S., Rivenq, A., Zighem, M. E. N., Hadid, A., Ouahabi, A., & Taleb-Ahmed, A. (2021). Automatic pain estimation from facial expressions: A comparative analysis using off-the-shelf CNN architectures. Electronics, 10(16), 1926. https://doi.org/10.3390/electronics10161926
Janssen, B. (2021). Pain by association: Role of individual difference variables. 2020–2021.
Kaltwang, S. (2015). Regression-based estimation of pain and facial expression intensity. Imperial College London, Department of Computing, 43(5), 127–137.
Leo, M., Carcagnì, P., Mazzeo, P. L., Spagnolo, P., Cazzato, D., & Distante, C. (2020). Analysis of facial information for healthcare applications: A survey on computer vision-based approaches. Information, 11(3), 128. https://doi.org/10.3390/info11030128
Nour, N., Elhebir, M., & Viriri, S. (2020). Face expression recognition using convolution neural network (CNN) models. International Journal of Grid Computing and Applications, 11(4), 1–11. https://doi.org/10.5121/ijgca.2020.11401
Raja, S. N., et al. (2020). The revised International Association for the Study of Pain definition of pain: Concepts, challenges, and compromises. Pain, 161(9), 1976–1982. https://doi.org/10.1097/j.pain.0000000000001939
Saddam, E., Mutashar, S., & Ali, W. (2021). A study of patient’s pain assessment based on facial expression: Issues and challenges. Engineering Technology Journal, 39(10), 1514–1527. https://doi.org/10.30684/etj.v39i10.2079
Schroff, F., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. arXiv.
Shier, W. A. (2017). Automated pain recognition using analysis of facial expressions. https://doi.org/10.11575/PRISM/25076
Virrey, R. A., De Silva Liyanage, C., Iskandar bin Pg Hj Petra, M., & Emeroylariffion Abas, P. (2019). Visual data of facial expressions for automatic pain detection. Journal of Visual Communication and Image Representation, 61, 209–217. https://doi.org/10.1016/j.jvcir.2019.03.023