The aim of this study was to compare three web application test automation tools: Playwright, Cypress and Selenium. The comparative analysis was carried out using an empirical study, which consisted of executing test scenarios and reviewing the documentation of the tested tools. The following measures were used as criteria for comparison: average test execution time, average percentage CPU usage, average percentage RAM usage. Based on the results obtained and their analysis, it was concluded that Playwright has shown higher effectiveness and flexibility in most test cases.
References
[1] S. Butt, S. U. R. Khan, S. Hussain, W. Wang, A conceptual model supporting decision-making for test automation in Agile-based Software Development, Data & Knowledge Engineering 144 (2023) 102–111, https://doi.org/10.1016/j.datak.2022.102111.
[2] Y. Wang, M.V. Mäntylä, Z. Liu, J. Markkula, Test automation maturity improves product quality – Quantitative study of open source projects using continuous integration, Journal of Systems and Software 188 (2022) 1-39, https://doi.org/10.48550/arXiv.2202.04068.
[3] D. Zimmermann, A. Koziolek, GUI-Based Software Testing: An Automated Approach Using GPT-4 and Selenium WebDriver, In 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) (2023) 171-174, https://doi.org/10.1109/ASEW60602.2023.00028.
[4] B. Jose, Test Automation; A manager’s guide, BCS Learning and Development, Swindon, 2021.
[5] D. Zhyhulin, K. Kasian, M. Kasian, Combined method of prioritization and automation of software regression testing, In IEEE 16th International Conference on Advanced Trends in Radioelectronics, Telecommunications and Computer Engineering (TCSET) (2022) 751-755, https://doi.org/10.1109/TCSET55632.2022.9767034.
[6] S. Gojare, R. Joshi, D. Gaigaware, Analysis and Design of Selenium WebDriver Automation Testing Framework, Procedia Computer Science 50 (2015) 341–346, https://doi.org/10.1016/j.procs.2015.04.038.
[7] E. Pelivani, A. Besimi, B. Cico, A Comparative Study of UI Testing Framework, In 2022 11th Mediterranean Conference on Embedded Computing (MECO) (2022) 1-5, https://doi.org/10.1109/MECO55406.2022.9797165.
[8] A. Utomo, G. Wijaya, Y. Setiawan, Implementation of crowdfunding web application using AWS Amplify architecture with end-to-end testing using Playwright, Indonesian Journal of Multidisciplinary Science 11 (2023) 3888–3904, https://doi.org/10.55324/ijoms.v2i11.620.
[9] R. Angmo, M. Sharma, Performance Evaluation of Web Based Automation Testing Tools, In International Conference on Confluence The Next Generation Information Technology Summit (Confluence) (2014) 731-735, https://doi.org/10.1109/CONFLUENCE.2014.6949287.
[10] R. Gupta, Ultimate Selenium WebDriver for Test Automation, Orange Education Pvt, Delhi, 2024.
[11] W. Mwaura, End-to-End Web Testing with Cypress, Packt Publishing, Birmingham, 2021.
[12] K. Rahman, Science of Selenium, BPB Publications, Noida, 2020.
[13] M. Leotta, F. Ricca, A. Marchetto, D. Olianas, An empirical study to compare three web test automation approaches: NLP-based, programmable, and capture&replay, Journal of Software: Evolution and Process 36(5) (2024) 1-24, https://doi.org/10.1002/smr.2606.
[14] M. Sambamurthy, Test Automation Engineering Handbook, Packt Publishing, Birmingham, 2023.
[15] A. Axelrod, Complete Guide to Test Automation, Apress, Berkeley, 2018.
[16] Open-source web application used in the paper, https://practicesoftwaretesting.com, [29.12.2024].
[17] GitHub repository with source code of the used application, https://github.com/testsmith-io/practice-software-testing, [29.12.2024].
[18] Front page of Visual Studio Code IDE website, https://code.visualstudio.com/, [29.12.2024].
[19] Overview section of technical documentation of Performance Monitor, https://learn.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-r2-and-2008/cc749154(v=ws.11), [29.12.2024].
[20] Front page of npm website, https://www.npmjs.com/, [29.12.2024].
[21] Front page of Node.js runtime website, https://nodejs.org/en, [29.12.2024].
[22] Front page of npm trends website, https://npmtrends.com/, [29.12.2024].