Comparison of tools for creating and conducting automated tests

Grzegorz Wojciech Bielesza

grzegorz.bielesza@pollub.edu.pl
Lublin University of Technology (Poland)

Mariusz Dzieńkowski


Lublin University of Technology
https://orcid.org/0000-0002-1932-297X

Abstract

The presented work includes a comparison of three tools for preparing and conducting automated tests - JUnit5, TestNG and Spock. The study, on the basis of which the comparative analysis was carried out, consisted in performing three types of experiments, i.e. performance tests based on five test scenarios, functional tests checking the capabilities of each tool, and tests of popularity and support in the modern context. The performance experiment consisted in configuring the test environment, preparing tests based on scenarios, performing them multiple times, and then verifying and analysing the obtained time results. The comparison of tools capabilities was based on the preparation of a set of key functionalities of the software used to develop tests. The last analysis consisted in comparing the popularity of the surveyed platforms, and was performed on the basis of three popular websites providing information on user activity within the test tool.


Keywords:

automated testing, JUnit5, TestNG, Spock

R. Torkar, Towards Automated Software Testing: Techniques, Classifications and Frameworks (PhD dissertation, Blekinge Institute of Technology) (2006).
  Google Scholar

I. B. K. Manuaba, Combination of test-driven development and behavior-driven development for improving backend testing performance. Procedia Computer Science 157 (2019) 79-86.
DOI: https://doi.org/10.1016/j.procs.2019.08.144   Google Scholar

M. Sharma, R. Angmo, Web based automation testing and tools. International Journal of Computer Science and Information Technologies 5(1) (2014) 908-912.
  Google Scholar

M. A. Umar, C. Zhanfang, A study of automated software testing: Automation tools and frameworks. International Journal of Computer Science Engineering (IJCSE) 6 (2019) 217-225.
  Google Scholar

A. Orso, Integration testing of object-oriented software. Dottorato di Ricerca in Ingegneria Informatica e Automatica, Politecnico di Milano, 1998.
  Google Scholar

D. Gromek, D. Gutek, Analysis and comparison of programming frameworks used for automated tests. Journal of Computer Sciences Institute 17 (2020) 339-344.
DOI: https://doi.org/10.35784/jcsi.2097   Google Scholar

M. Jain, D. Gopalani, Automated Java testing: JUnit versus AspectJ. International Journal of Computer and Information Engineering 11(11) (2017) 1215-1220.
  Google Scholar

A. Shtokal, J. Smołka, Comparative analysis of frameworks used in automated testing on example of TestNG and WebdriverIO. Journal of Computer Sciences Institute 19 (2021) 100-106.
DOI: https://doi.org/10.35784/jcsi.2595   Google Scholar

H. Kaur, G. Gupta, Comparative study of automated testing tools: selenium, quick test professional and testcomplete. Int. Journal of Engineering Research and Applications 3(5) (2013) 1739-1743.
  Google Scholar

Download


Published
2023-12-29

Cited by

Bielesza, G. W., & Dzieńkowski, M. (2023). Comparison of tools for creating and conducting automated tests. Journal of Computer Sciences Institute, 29, 374–382. https://doi.org/10.35784/jcsi.3804

Authors

Grzegorz Wojciech Bielesza 
grzegorz.bielesza@pollub.edu.pl
Lublin University of Technology Poland

Authors

Mariusz Dzieńkowski 

Lublin University of Technology
https://orcid.org/0000-0002-1932-297X

Statistics

Abstract views: 103
PDF downloads: 134