EXAMINATION OF SUMMARIZED MEDICAL RECORDS FOR ICD CODE CLASSIFICATION VIA BERT
Article Sidebar
Open full text
Issue Vol. 20 No. 2 (2024)
-
FEW-SHOT LEARNING WITH PRE-TRAINED LAYERS INTEGRATION APPLIED TO HAND GESTURE RECOGNITION FOR DISABLED PEOPLE
Mohamed ELBAHRI, Nasreddine TALEB, Sid Ahmed El Mehdi ARDJOUN, Chakib Mustapha Anouar ZOUAOUI1-23
-
DIGITAL NEWS CLASSIFICATION AND PUNCTUACTION USING MACHINE LEARNING AND TEXT MINING TECHNIQUES
Fernando Andrés CEVALLOS SALAS24-42
-
MODELING THE OPTIMAL MEASUREMENT TIME WITH A PROBE ON THE MACHINE TOOL USING MACHINE LEARNING METHODS
Jerzy JÓZWIK, Magdalena ZAWADA-MICHAŁOWSKA, Monika KULISZ, Paweł TOMIŁO, Marcin BARSZCZ, Paweł PIEŚKO, Michał LELEŃ, Kamil CYBUL43-59
-
EXAMINATION OF SUMMARIZED MEDICAL RECORDS FOR ICD CODE CLASSIFICATION VIA BERT
Dilek AYDOGAN-KILIC, Deniz Kenan KILIC, Izabela Ewa NIELSEN60-74
-
THE UTILIZATION OF 6G IN INDUSTRY 4.0
Hanan M. SHUKUR, Shavan ASKAR, Subhi R.M. ZEEBAREE75-89
-
APPLICATION OF EEMD-DFA ALGORITHMS AND ANN CLASSIFICATION FOR DETECTION OF KNEE OSTEOARTHRITIS USING VIBROARTHROGRAPHY
Anna MACHROWSKA, Robert KARPIŃSKI, Marcin MACIEJEWSKI, Józef JONAK, Przemysław KRAKOWSKI90-108
-
PREDICTING STATES OF EPILEPSY PATIENTS USING DEEP LEARNING MODELS
Boutkhil SIDAOUI109-125
-
IMPROVING E-LEARNING BY FACIAL EXPRESSION ANALYSIS
Amina KINANE DAOUADJI, Fatima BENDELLA126-137
-
EXPLORING THE IMPACT OF ARTIFICIAL INTELLIGENCE ON HUMANROBOT COOPERATION IN THE CONTEXT OF INDUSTRY 4.0
Hawkar ASAAD, Shavan ASKAR, Ahmed KAKAMIN, Nayla FAIQ138-156
-
AN AUTHENTICATION METHOD BASED ON A DIOPHANTINE MODEL OF THE COIN BAG PROBLEM
Krzysztof NIEMIEC, Grzegorz BOCEWICZ157-174
-
PREDICTION OF PATIENT’S WILLINGNESS FOR TREATMENT OF MENTAL ILLNESS USING MACHINE LEARNING APPROACHES
Mohammed Chachan YOUNIS175-193
-
AUTOMATION OF POLYCYSTIC OVARY SYNDROME DIAGNOSTICS THROUGH MACHINE LEARNING ALGORITHMS IN ULTRASOUND IMAGING
Roman GALAGAN, Serhiy ANDREIEV, Nataliia STELMAKH; Yaroslava RAFALSKA; Andrii MOMOT194-204
Archives
-
Vol. 21 No. 3
2025-10-05 12
-
Vol. 21 No. 2
2025-06-27 12
-
Vol. 21 No. 1
2025-03-31 12
-
Vol. 20 No. 4
2025-01-31 12
-
Vol. 20 No. 3
2024-09-30 12
-
Vol. 20 No. 2
2024-08-14 12
-
Vol. 20 No. 1
2024-03-30 12
-
Vol. 19 No. 4
2023-12-31 10
-
Vol. 19 No. 3
2023-09-30 10
-
Vol. 19 No. 2
2023-06-30 10
-
Vol. 19 No. 1
2023-03-31 10
-
Vol. 18 No. 4
2022-12-30 8
-
Vol. 18 No. 3
2022-09-30 8
-
Vol. 18 No. 2
2022-06-30 8
-
Vol. 18 No. 1
2022-03-30 7
-
Vol. 16 No. 4
2020-12-30 8
-
Vol. 16 No. 3
2020-09-30 8
-
Vol. 16 No. 2
2020-06-30 8
-
Vol. 16 No. 1
2020-03-30 8
Main Article Content
DOI
Authors
Abstract
The International Classification of Diseases (ICD) is utilized by member countries of the World Health Organization (WHO). It is a critical system to ensure worldwide standardization of diagnosis codes, which enables data comparison and analysis across various nations. The ICD system is essential in supporting payment systems, healthcare research, service planning, and quality and safety management. However, the sophisticated and intricate structure of the ICD system can sometimes cause issues such as longer examination times, increased training expenses, a greater need for human resources, problems with payment systems due to inaccurate coding, and unreliable data in health research. Additionally, machine learning models that use automated ICD systems face difficulties with lengthy medical notes. To tackle this challenge, the present study aims to utilize Medical Information Mart for Intensive Care (MIMIC-III) medical notes that have been summarized using the term frequency-inverse document frequency (TF-IDF) method. These notes are further analyzed using deep learning, specifically bidirectional encoder representations from transformers (BERT), to classify disease diagnoses based on ICD codes. Even though the proposed methodology using summarized data provides lower accuracy performance than state-of-the-art methods, the performance results obtained are promising in terms of continuing the study of extracting summary input and more important features, as it provides real-time ICD code classification and more explainable inputs.
Keywords:
References
Alsentzer, E., Murphy, J. R., Boag, W., Weng, W. H., Jin, D., Naumann, T., & McDermott, M. (2019). Publicly available clinical BERT embeddings. arXiv preprint. https://doi.org/10.48550/arXiv.1904.03323 DOI: https://doi.org/10.18653/v1/W19-1909
Baumel, T., Nassour-Kassis, J., Cohen, R., Elhadad, M., & Elhadad, N. (2018, June). Multi-label classification of patient notes: case study on ICD code assignment. In Workshops at the thirty-second AAAI conference on artificial intelligence.
Bhargava, P., Drozd, A., & Rogers, A. (2021). Generalization in NLI: Ways (not) to go beyond simple heuristics. arXiv preprint. https://doi.org/10.48550/arXiv.2110.01518 DOI: https://doi.org/10.18653/v1/2021.insights-1.18
Cao, P., Chen, Y., Liu, K., Zhao, J., Liu, S., & Chong, W. (2020a, July). HyperCore: Hyperbolic and co-graph representation for automatic ICD coding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3105-3114. https://doi.org/10.18653/v1/2020.acl-main.282 DOI: https://doi.org/10.18653/v1/2020.acl-main.282
Cao, P., Yan, C., Fu, X., Chen, Y., Liu, K., Zhao, J., Liu, S., & Chong, W. (2020b, July). Clinical-coder: Assigning interpretable ICD-10 codes to Chinese clinical notes. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 294-301. https://doi.org/10.18653/v1/2020.acl-demos.33 DOI: https://doi.org/10.18653/v1/2020.acl-demos.33
Chen, P. F., Wang, S. M., Liao, W. C., Kuo, L. C., Chen, K. C., Lin, Y. C., Yang, C., Chiu, C., Chang, S., & Lai, F. (2021). Automatic ICD-10 coding and training system: deep neural network based on supervised learning. JMIR Medical Informatics, 9(8), e23230. https://doi.org/10.2196/23230 DOI: https://doi.org/10.2196/23230
Chute, C. G., & Çelik, C. (2021). Overview of ICD-11 architecture and structure. BMC medical informatics and decision making, 21(6), 1-7. https://doi.org/10.1186/s12911-021-01539-1 DOI: https://doi.org/10.1186/s12911-021-01539-1
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint. https://doi.org/10.48550/arXiv.1810.04805
Du, Y., Xu, T., Ma, J., Cen, E., Zheng, Y., Liu, T., & Tong, G. (2020). An automatic ICD coding method for clinical records based on deep neural network. Big Data Res, 6(5), 3-15. https://doi.org/10.11959/j.issn.2096-0271.2020040
Farkas, R., & Szarvas, G. (2008). Automatic construction of rule-based ICD-9-CM coding systems. BMC bioinformatics, 9 Suppl 3(Suppl 3), S10. https://doi.org/10.1186/1471-2105-9-S3-S10 DOI: https://doi.org/10.1186/1471-2105-9-S3-S10
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., Mietus, J. E., Moody, G. B., Peng, C. K., & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation [Online]. 101 (23), pp. e215–e220. DOI: https://doi.org/10.1161/01.CIR.101.23.e215
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J. & Poon, H. (2021). Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1), 1-23. https://doi.org/10.1145/3458754 DOI: https://doi.org/10.1145/3458754
Harrison, J. E., Weber, S., Jakob, R., & Chute, C. G. (2021). ICD-11: an international classification of diseases for the twenty-first century. BMC medical informatics and decision making, 21(6), 1-10. https://doi.org/10.1186/s12911-021-01534-6 DOI: https://doi.org/10.1186/s12911-021-01534-6
Hsu, J. L., Hsu, T. J., Hsieh, C. H., & Singaravelan, A. (2020). Applying convolutional neural networks to predict the ICD-9 codes of medical records. Sensors, 20(24), 7116. https://doi.org/10.3390/s20247116 DOI: https://doi.org/10.3390/s20247116
Huang, C. W., Tsai, S. C., & Chen, Y. N. (2022). PLM-ICD: automatic ICD coding with pretrained language models. arXiv preprint. https://doi.org/10.48550/arXiv.2207.05289 DOI: https://doi.org/10.18653/v1/2022.clinicalnlp-1.2
Huang, J., Osorio, C., & Sy L. W. (2019). An empirical evaluation of deep learning for ICD-9 code assignment using MIMIC-III clinical notes. Computer Methods and Programs in Biomedicine, 177, 141–153. https://doi.org/10.1016/j.cmpb.2019.05.024 DOI: https://doi.org/10.1016/j.cmpb.2019.05.024
Johnson, A., Pollard, T., & Mark, R. (2016a). MIMIC-III Clinical Database (version 1.4). PhysioNet. https://doi.org/10.13026/C2XW26
Johnson, A. E., Pollard, T. J., Shen, L., Lehman, L. W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Celi, L. A., & Mark, R. G. (2016b). MIMIC-III, a freely accessible critical care database. Scientific data, 3(1), 1-9. https://doi.org/10.1038/sdata.2016.35 DOI: https://doi.org/10.1038/sdata.2016.35
Kaur, R., & Ginige, J. A. (2018). Comparative analysis of algorithmic approaches for auto-coding with ICD-10-AM and ACHI. Studies in health technology and informatics, 252, 73-79. https://doi.org/10.3233/978-1-61499-890-7-73
Kaur, R., Ginige, J. A., & Obst, O. (2021). A systematic literature review of automated ICD coding and classification systems using discharge summaries. arXiv preprint. https://doi.org/10.48550/arXiv.2107.10652
Li, F., & Yu, H. (2020, April). ICD coding from clinical text using multi-filter residual convolutional neural network. In proceedings of the AAAI conference on artificial intelligence, 34(05), pp. 8180-8187. https://doi.org/10.1609/aaai.v34i05.6331 DOI: https://doi.org/10.1609/aaai.v34i05.6331
Li, M., Fei, Z., Zeng, M., Wu, F. X., Li, Y., Pan, Y., & Wang, J. (2019). Automated ICD-9 coding via a deep learning approach. IEEE/ACM transactions on computational biology and bioinformatics, 16(4), 1193-1202. https://doi.org/10.1109/TCBB.2018.2817488 DOI: https://doi.org/10.1109/TCBB.2018.2817488
Marafino, B. J., Davies, J. M., Bardach, N. S., Dean, M. L., & Dudley, R. A. (2014). N-gram support vector machines for scalable procedure and diagnosis classification, with applications to clinical free text data from the intensive care unit. Journal of the American Medical Informatics Association, 21(5), 871-875. https://doi.org/10.1136/amiajnl-2014-002694 DOI: https://doi.org/10.1136/amiajnl-2014-002694
Minh, D., Wang, H. X., Li, Y. F., & Nguyen, T. N. (2022). Explainable artificial intelligence: a comprehensive review. Artificial Intelligence Review, 1-66. https://doi.org/10.1007/s10462-021-10088-y DOI: https://doi.org/10.1007/s10462-021-10088-y
Moons, E., Khanna, A., Akkasi, A., & Moens, M. F. (2020). A comparison of deep learning methods for ICD coding of clinical records. Applied Sciences, 10(15), 5262. https://doi.org/10.3390/app10155262 DOI: https://doi.org/10.3390/app10155262
Mullenbach, J., Wiegreffe, S., Duke, J., Sun, J., & Eisenstein, J. (2018). Explainable Prediction of Medical Codes from Clinical Text. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1101–1111, New Orleans, Louisiana. Association for Computational Linguistics. https://doi.org/10.18653/v1/N18-1100 DOI: https://doi.org/10.18653/v1/N18-1100
Nawalkar, N., Attar, V. Z., & Kalamkar, S. P. (2022). Automated icd-9 medical code assignment from given free text using deep learning approach. In Advances in Data and Information Sciences: Proceedings of ICDIS 2021 (pp. 317-327). Singapore: Springer Singapore. https://doi.org/10.1007/978-981-16-5689-7_28 DOI: https://doi.org/10.1007/978-981-16-5689-7_28
Pascual, D., Luck, S., & Wattenhofer, R. (2021). Towards BERT-based automatic ICD coding: Limitations and opportunities. arXiv preprint. https://doi.org/10.48550/arXiv.2104.06709 DOI: https://doi.org/10.18653/v1/2021.bionlp-1.6
Perotte, A., Pivovarov, R., Natarajan, K., Weiskopf, N., Wood, F., & Elhadad, N. (2014). Diagnosis code assignment: models and evaluation metrics. Journal of the American Medical Informatics Association, 21(2), 231-237. https://doi.org/10.1136/amiajnl-2013-002159 DOI: https://doi.org/10.1136/amiajnl-2013-002159
Pezzella, P. (2022). The ICD‐11 is now officially in effect. World Psychiatry, 21(2), 331.8. https://doi.org/10.1002/wps.20982 DOI: https://doi.org/10.1002/wps.20982
Ponthongmak, W., Thammasudjarit, R., McKay, G. J., Attia, J., Theera-Ampornpunt, N., & Thakkinstian, A. (2023). Development and external validation of automated ICD-10 coding from discharge summaries using deep learning approaches. Informatics in Medicine Unlocked, 38, 101227. https://doi.org/10.1016/j.imu.2023.101227 DOI: https://doi.org/10.1016/j.imu.2023.101227
Rios, A., & Kavuluru, R. (2018). Few-Shot and Zero-Shot Multi-Label Learning for Structured Label Spaces. Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, 2018, 3132–3142. NIH Public Access. DOI: https://doi.org/10.18653/v1/D18-1352
Scheurwegs, E., Luyckx, K., Luyten, L., Daelemans, W., & Van den Bulcke, T. (2016). Data integration of structured and unstructured sources for assigning clinical codes to patient stays. Journal of the American Medical Informatics Association, 23(e1), e11-e19. https://doi.org/10.1093/jamia/ocv115 DOI: https://doi.org/10.1093/jamia/ocv115
Shi, H., Xie, P., Hu, Z., Zhang, M., & Xing, E. P. (2017). Towards automated ICD coding using deep learning. arXiv preprint. https://doi.org/10.48550/arXiv.1711.04075
Singaravelan, A., Hsieh, C. H., Liao, Y. K., & Hsu, J. L. (2021). Predicting icd-9 codes using self-report of patients. Applied Sciences, 11(21), 10046. https://doi.org/10.3390/app112110046 DOI: https://doi.org/10.3390/app112110046
Tabassum, A., & Patil, R. R. (2020). A survey on text pre-processing & feature extraction techniques in natural language processing. International Research Journal of Engineering and Technology (IRJET), 7(06), 4864-4867.
Teng, F., Liu, Y., Li, T., Zhang, Y., Li, S., & Zhao, Y. (2022). A review on deep neural networks for ICD coding. IEEE Transactions on Knowledge and Data Engineering, 35(5), 4357-4375. https://doi.org/10.1109/TKDE.2022.3148267 DOI: https://doi.org/10.1109/TKDE.2022.3148267
Turc, I., Chang, M. W., Lee, K., & Toutanova, K. (2019). Well-read students learn better: The impact of student initialization on knowledge distillation. arXiv preprint. https://doi.org/10.48550/arXiv.1908.08962
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30. Curran Associates, Inc.
Vu, T., Nguyen, D. Q., & Nguyen, A. (2020). A label attention model for ICD coding from clinical text. arXiv preprint. https://doi.org/10.48550/arXiv.2007.06351 DOI: https://doi.org/10.24963/ijcai.2020/461
Wang, D., Su, J., & Yu, H. (2020). Feature Extraction and Analysis of Natural Language Processing for Deep Learning English Language. IEEE Access, 8, 46335-46345. https://doi.org/10.1109/ACCESS.2020.2974101 DOI: https://doi.org/10.1109/ACCESS.2020.2974101
Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Henao, R., & Carin, L. (2018). Joint embedding of words and labels for text classification. arXiv preprint. https://doi.org/10.48550/arXiv.1805.04174 DOI: https://doi.org/10.18653/v1/P18-1216
Wu, X., Zhao, Y., Yang, Y., Liu, Z., & Clifton, D. A. (2022). A Comparison of Representation Learning Methods for Medical Concepts in MIMIC-IV. medRxiv, 2022-08. http://dx.doi.org/10.2139/ssrn.4583878 DOI: https://doi.org/10.1101/2022.08.21.22278835
Yan, C., Fu, X., Liu, X., Zhang, Y., Gao, Y., Wu, J., & Li, Q. (2022). A survey of automated International Classification of Diseases coding: development, challenges, and applications. Intelligent Medicine, 2(3), 161-173. https://doi.org/10.1016/j.imed.2022.03.003 DOI: https://doi.org/10.1016/j.imed.2022.03.003
Zeng, M., Li, M., Fei, Z., Yu, Y., Pan, Y., & Wang, J. (2019). Automatic ICD-9 coding via deep transfer learning. Neurocomputing, 324, 43-50. https://doi.org/10.1016/j.neucom.2018.04.081 DOI: https://doi.org/10.1016/j.neucom.2018.04.081
Zhang, Z., Liu, J., & Razavian, N. (2020). BERT-XML: Large scale automated ICD coding using BERT pretraining. arXiv preprint. https://doi.org/10.48550/arXiv.2006.03685 DOI: https://doi.org/10.18653/v1/2020.clinicalnlp-1.3
Article Details
Abstract views: 591
License

This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
