APPLICATION OF EXPLAINABLE ARTIFICIAL INTELLIGENCE IN SOFTWARE BUG CLASSIFICATION
Łukasz Chmielowski
lukasz.chmielowski@nokia.comNokia Solutions and Networks sp. z o.o. (Poland)
http://orcid.org/0000-0001-6970-8144
Michał Kucharzak
Nokia Solutions and Networks sp. z o.o. (Poland)
https://orcid.org/0000-0001-5068-5229
Robert Burduk
Wroclaw University of Science and Technology (Poland)
https://orcid.org/0000-0002-3506-6611
Abstract
Fault management is an expensive process and analyzing data manually requires a lot of resources. Modern software bug tracking systems may be armed with automated bug report assignment functionality that facilitates bug classification or bug assignment to proper development group. For supporting decision systems, it would be beneficial to introduce information related to explainability. The purpose of this work is to evaluate the use of explainable artificial intelligence (XAI) in processes related to software development and bug classification based on bug reports created by either software testers or software users. The research was conducted on two different datasets. The first one is related to classification of security vs non-security bug reports. It comes from a telecommunication company which develops software and hardware solutions for mobile operators. The second dataset contains a list of software bugs taken from an opensource project. In this dataset the task is to classify issues with one of following labels crash, memory, performance, and security. Studies on XAI-related algorithms show that there are no major differences in the results of the algorithms used when comparing them with others. Therefore, not only the users can obtain results with possible explanations or experts can verify model or its part before introducing into production, but also it does not provide degradation of accuracy. Studies showed that it could be put into practice, but it has not been done so far.
Keywords:
software bug assignment, software bug triaging, explainable artificial intelligence, text analysis, vulnerabilityReferences
Aleithan R.: Explainable Just-In-Time Bug Prediction: Are We There Yet? 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), 2021, 129–131 [http://doi.org/10.1109/ICSE-Companion52605.2021.00056].
DOI: https://doi.org/10.1109/ICSE-Companion52605.2021.00056
Google Scholar
Anjali, Mohan D., Sardana N.: Visheshagya: Time based expertise modelfor bug report assignment. Ninth International Conference on Contemporary Computing (IC3), 2016, 1–6 [http://doi.org/10.1109/IC3.2016.7880218].
DOI: https://doi.org/10.1109/IC3.2016.7880218
Google Scholar
Barredo Arrieta A. et al.: Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58, 2020, 82–115 [http://doi.org/10.1016/j.inffus.2019.12.012].
DOI: https://doi.org/10.1016/j.inffus.2019.12.012
Google Scholar
Behl D., Handa S., Arora A.: A bug Mining tool to identify and analyze security bugs using Naive Bayes and TF-IDF. International Conference on Reliability Optimization and Information Technology (ICROIT), 2014, 294–299 [http://doi.org/10.1109/ICROIT.2014.6798341].
DOI: https://doi.org/10.1109/ICROIT.2014.6798341
Google Scholar
Carlevaro A., Maurizio M.: A New SVDD Approach to Reliable and Explainable AI. IEEE Intelligent Systems 37.2, 2022, 55–68 [http://doi.org/10.1109/ACCESS.2022.3180026].
DOI: https://doi.org/10.1109/MIS.2021.3123669
Google Scholar
Carlevaro A. et al.: Counterfactual Building and Evaluation via eXplainable Support Vector Data Description. IEEE Access 10, 2022, [http://doi.org/10.1109/MIS.2021.3123669].
DOI: https://doi.org/10.1109/ACCESS.2022.3180026
Google Scholar
Castelluccio M. et al.: bugbug. Available online: https://github.com/mozilla/bugbug (accessed on 02.11.2022).
Google Scholar
Chmielowski L., Kucharzak M.: Impact of Software Bug Report Preprocessing and ectorization on Bug Assignment Accuracy. Progress in Image Processing, Pattern Recognition and Communication Systems. Edited by Michal Choraś, et al.: Springer International Publishing, Cham 2022, 153–162 [http://doi.org/10.1007/978-3-030-81523-3_15].
DOI: https://doi.org/10.1007/978-3-030-81523-3_15
Google Scholar
Choquette-Choo C. A. et al.: A Multi-label, Dual-Output Deep Neural Network for Automated Bug Triaging. 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 2019, 937–944, [http://doi.org/10.1109/ICMLA.2019.00161].
DOI: https://doi.org/10.1109/ICMLA.2019.00161
Google Scholar
Gujral S., et al.: Classifying bug severity using dictionary based approach. International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE), 2015, 599–602, [http://doi.org/10.1109/ABLAZE.2015.7154933].
DOI: https://doi.org/10.1109/ABLAZE.2015.7154933
Google Scholar
Khanan C. et al.: JITBot: An Explainable Just-In-Time Defect Prediction Bot. 35th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2020, 1336–1339.
DOI: https://doi.org/10.1145/3324884.3415295
Google Scholar
Lamkanfi A., Pérez J., Demeyer S.: The Eclipse and Mozilla defect tracking dataset: A genuine dataset for mining bug information. 10th Working Conference on Mining Software Repositories (MSR), 2013, 203–206 [http://doi.org/10.1109/MSR.2013.6624028].
DOI: https://doi.org/10.1109/MSR.2013.6624028
Google Scholar
Matzka S.: Explainable Artificial Intelligence for Predictive Maintenance Applications. Third International Conference on Artificial Intelligence for Industries (AI4I), 2020, 69–74, [https://doi.org/10.1109/AI4I49448.2020.00023].
DOI: https://doi.org/10.1109/AI4I49448.2020.00023
Google Scholar
Monperrus M.: Explainable Software Bot Contributions: Case Study of Automated Bug Fixes. IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE), 2019, 12–15, [http://doi.org/10.1109/BotSE.2019.00010].
DOI: https://doi.org/10.1109/BotSE.2019.00010
Google Scholar
Raschka S.: 5x2cv paired ttest. Available online: https://rasbt.github.io/mlxtend/user_guide/evaluate/paired_ttest_5x2cv (accessed on 04.01.2021).
DOI: https://doi.org/10.1093/noajnl/vdaa143.020
Google Scholar
Vilone G. Longo L.: Explainable Artificial Intelligence: a Systematic Review. 2020, [http://doi.org/10.48550/arXiv.2006.00093].
Google Scholar
Authors
Łukasz Chmielowskilukasz.chmielowski@nokia.com
Nokia Solutions and Networks sp. z o.o. Poland
http://orcid.org/0000-0001-6970-8144
Łukasz Chmielowski received a M.Sc. degree with distinction in Computer Science with specialization in Intelligent Information Systems. He is currently working towards Ph.D. in Information and Communication Technology at Wroclaw University of Science and Technology (Poland). He is with the Nokia Solutions and Networks sp. z o.o. (Poland) for five years. He is working with machine learning techniques related to natural language processing and software bug assignment.
Authors
Michał KucharzakNokia Solutions and Networks sp. z o.o. Poland
https://orcid.org/0000-0001-5068-5229
Michał Kucharzak received his Ph.D. in computer science in area of network optimization. In recent years, he cooperated with numerous R&D centers and has been a member of reviewer committees for many international journals, program, and technical committees for various conferences as well. His current research interests are primarily in the areas of network modeling and network optimization with special regard to overlays, simulations, design of efficient algorithms and wireless system protocols, including software testing and quality assurance.
Authors
Robert BurdukWroclaw University of Science and Technology Poland
https://orcid.org/0000-0002-3506-6611
Robert Burduk is Professor of Computer Science in the Department of Systems and Computer Networks, Faculty of Information and Communication Technology, Wroclaw University of Science and Technology, Poland. He received an Ph.D. and D.Sc. degrees in Computer Science in 2003 and 2014 respectively. His research interests cover among the others: machine learning, classifier selection algorithms and multiple classifier systems. He serves on program committees of numerous international conferences, published over 100 papers and edited 5 books.
Statistics
Abstract views: 600PDF downloads: 338
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
