Analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment

Article Sidebar

Open full text

PDF
Published: Mar 20, 2024
DOI: https://doi.org/10.35784/jcsi.4060
Crossref
Scopus
Google Scholar
Europe PMC
Articles

Archives

2024

Main Article Content

DOI

https://doi.org/10.35784/jcsi.4060

Authors

Mikołaj Skrzypczyński

mikolaj.skrzypczynski@pollub.edu.pl

Lublin University of Technology, Poland
Piotr Muryjas

p.muryjas@pollub.pl

Lublin University of Technology, Poland

Abstract


The aim of this paper is the analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment. The analysis was based on comparison between both mentioned tools with use of large data set, represented by 28 million records. Research was provided with use of scripts and queries destined for Apache Hive and Apache Pig, and then executed 10 times on environment brought by created virtual machine. Those methods were performed on the same data sets for 16 times according to previously prepared research scenarios. As the conclusion, authors had observed that Apache Hive is more efficient tool, than Apache Pig.


Keywords:

data processing, Apache Hive, Apache Pig, Hadoop

References

Article Details

Skrzypczyński, M., & Muryjas, P. (2024). Analysis of data processing efficiency with use of Apache Hive and Apache Pig in Hadoop environment. Journal of Computer Sciences Institute, 30, 1–8. https://doi.org/10.35784/jcsi.4060