Performance analysis of the TensorFlow library with different optimisation algorithms
Maciej Wadas
maciej.wadas@pollub.edu.plPolitechnika Lubelska Wydział Elektrotechniki i Informatyki (Poland)
Jakub Smołka
Lublin University of Technology (Poland)
Abstract
This paper presents the results of performance analysis of the Tensorflow library used in machine learning and deep neural networks. The analysis focuses on comparing the parameters obtained when training the neural network model for optimization algorithms: Adam, Nadam, AdaMax, AdaDelta, AdaGrad. Special attention has been paid to the differences between the training efficiency on tasks using microprocessor and graphics card. For the study, neural network models were created in order to recognise Polish handwritten characters. The results obtained showed that the most efficient algorithm is AdaMax, while the computer component used during the research only affects the training time of the neural network model used.
Keywords:
machine learning; neural networksReferences
J. McCarthy, From here to human-level AI, Artificial Intelligence 171 (2007) 1174–1182, https://doi.org/10.1016/j.artint.2007.10.009.
DOI: https://doi.org/10.1016/j.artint.2007.10.009
Google Scholar
T. Okuda, S. Shoda, AI-based chatbot service for financial industry, Fujitsu Scientific and Technical Journal 54 (2018) 4–8.
Google Scholar
S. Green, J. Heer, C. D. Manning, Natural language translation at the intersection of AI and HCI, Communications of the ACM 58 (2015) 46–53, https://doi.org/10.1145/2767151.
DOI: https://doi.org/10.1145/2767151
Google Scholar
K. Chakraborty, A. Talele, S. Upadhya, Voice recognition using MFCC algorithm, International Journal of Innovative Research in Advanced Engineering (IJIRAE) 1 (2014) 158–161.
Google Scholar
H. Fujiyoshi, T. Hirakawa, T. Yamashita, Deep learning-based image recognition for autonomous driving, IATSS research 43 (2019) 244–252, https://doi.org/10.1016/j.iatssr.2019.11.008.
DOI: https://doi.org/10.1016/j.iatssr.2019.11.008
Google Scholar
A. Abraham, F. Pedregosa, M. Eickenberg, P. Gervais, A. Mueller, J. Kossaifi, A. Gramfort, B. Thirion, G. Varoquaux, Machine learning for neuroimaging with scikit-learn, Frontiers in neuroinformatics 8 (2014) 1–14, https://doi.org/10.3389/fninf.2014.00014.
DOI: https://doi.org/10.3389/fninf.2014.00014
Google Scholar
J. Moolayil, J. Moolayil, S. John, Learn Keras for Deep Neural Networks, Birmingham: Apress, 2019.
DOI: https://doi.org/10.1007/978-1-4842-4240-7
Google Scholar
R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer et al., Theano: A Python framework for fast computation of mathematical expressions, Computing Research Repository (2016) 1–19.
Google Scholar
G. Zaccone, M. R. Karim, Deep learning with TensorFlow: Explore neural networks and build intelligent systems with python, Packt Publishing Ltd, 2018.
Google Scholar
S. Bahrampour, N. Ramakrishnan, L. Schott, M. Shah, Comparative study of deep learning software frameworks, Computing Research Repository (2015).
Google Scholar
S. Raschka, Python. Uczenie maszynowe, Packt Publishing Ltd, 2017.
Google Scholar
Firmy wykorzystujące bibliotekę Tensorflow, https://www.tensorflow.org/about/case-studies, [26.06.2021].
Google Scholar
Opis architektury biblioteki Tensorflow, https://developers.googleblog.com/2017/09/introducing-tensorflow-datasets.html, [26.06.2021].
Google Scholar
D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, International Conference on Learning Representations (2015) 1–15.
Google Scholar
T. Dozat, Incorporating Nesterov Momentum into Adam, International Conference on Learning Representations (2016) 1–4.
Google Scholar
A. Lydia, S. Francis, Adagrad–an optimizer for stochastic gradient descent, International Journal of Information and Computing Science 6 (2019) 566–568.
Google Scholar
M. D. Zeiler, Adadelta: an adaptive learning rate method, Computing Research Repository (2012).
Google Scholar
M. Tokovarov, M. Kaczorowska, M. Miłosz, Development of Extensive Polish Handwritten Characters Database for Text Recognition Research, Advances in Science and Technology Research Journal 14 (2020) 30–38, https://doi.org/10.12913/22998624/122567.
DOI: https://doi.org/10.12913/22998624/122567
Google Scholar
E. Lukasik, M. Charytanowicz, M. Milosz, M. Tokovarov, M. Kaczorowska, D. Czerwinski, T. Zientarski, Recognition of handwritten Latin characters with diacritics using CNN, Bulletin of the Polish Academy of Sciences: Technical Sciences 69 (2021) 1–12, http://dx.doi.org/10.24425/bpasts.2020.136210.
Google Scholar
E. Zitzler, Evolutionary algorithms for multiobjective optimization: Methods and applications, Ithaca: Shaker. 1999.
Google Scholar
K. Kukuła, Metoda unitaryzacji zerowanej na tle wybranych metod normowania cech diagnostycznych, Acta Scientifica Academiae Ostroviensis 4 (1999) 5–31.
Google Scholar
Authors
Maciej Wadasmaciej.wadas@pollub.edu.pl
Politechnika Lubelska Wydział Elektrotechniki i Informatyki Poland
Authors
Jakub SmołkaLublin University of Technology Poland
Statistics
Abstract views: 255PDF downloads: 252
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.