Performance analysis of the TensorFlow library with different optimisation algorithms

Maciej Wadas

maciej.wadas@pollub.edu.pl
Politechnika Lubelska Wydział Elektrotechniki i Informatyki (Poland)

Jakub Smołka


Lublin University of Technology (Poland)

Abstract

This paper presents the results of performance analysis of the Tensorflow library used in machine learning and deep neural networks. The analysis focuses on comparing the parameters obtained when training the neural network model for optimization algorithms: Adam, Nadam, AdaMax, AdaDelta, AdaGrad. Special attention has been paid to the differences between the training efficiency on tasks using microprocessor and graphics card. For the study, neural network models were created in order to recognise Polish handwritten characters. The results obtained showed that the most efficient algorithm is AdaMax, while the computer component used during the research only affects the training time of the neural network model used.


Keywords:

machine learning; neural networks

J. McCarthy, From here to human-level AI, Artificial Intelligence 171 (2007) 1174–1182, https://doi.org/10.1016/j.artint.2007.10.009.
DOI: https://doi.org/10.1016/j.artint.2007.10.009   Google Scholar

T. Okuda, S. Shoda, AI-based chatbot service for financial industry, Fujitsu Scientific and Technical Journal 54 (2018) 4–8.
  Google Scholar

S. Green, J. Heer, C. D. Manning, Natural language translation at the intersection of AI and HCI, Communications of the ACM 58 (2015) 46–53, https://doi.org/10.1145/2767151.
DOI: https://doi.org/10.1145/2767151   Google Scholar

K. Chakraborty, A. Talele, S. Upadhya, Voice recognition using MFCC algorithm, International Journal of Innovative Research in Advanced Engineering (IJIRAE) 1 (2014) 158–161.
  Google Scholar

H. Fujiyoshi, T. Hirakawa, T. Yamashita, Deep learning-based image recognition for autonomous driving, IATSS research 43 (2019) 244–252, https://doi.org/10.1016/j.iatssr.2019.11.008.
DOI: https://doi.org/10.1016/j.iatssr.2019.11.008   Google Scholar

A. Abraham, F. Pedregosa, M. Eickenberg, P. Gervais, A. Mueller, J. Kossaifi, A. Gramfort, B. Thirion, G. Varoquaux, Machine learning for neuroimaging with scikit-learn, Frontiers in neuroinformatics 8 (2014) 1–14, https://doi.org/10.3389/fninf.2014.00014.
DOI: https://doi.org/10.3389/fninf.2014.00014   Google Scholar

J. Moolayil, J. Moolayil, S. John, Learn Keras for Deep Neural Networks, Birmingham: Apress, 2019.
DOI: https://doi.org/10.1007/978-1-4842-4240-7   Google Scholar

R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer et al., Theano: A Python framework for fast computation of mathematical expressions, Computing Research Repository (2016) 1–19.
  Google Scholar

G. Zaccone, M. R. Karim, Deep learning with TensorFlow: Explore neural networks and build intelligent systems with python, Packt Publishing Ltd, 2018.
  Google Scholar

S. Bahrampour, N. Ramakrishnan, L. Schott, M. Shah, Comparative study of deep learning software frameworks, Computing Research Repository (2015).
  Google Scholar

S. Raschka, Python. Uczenie maszynowe, Packt Publishing Ltd, 2017.
  Google Scholar

Firmy wykorzystujące bibliotekę Tensorflow, https://www.tensorflow.org/about/case-studies, [26.06.2021].
  Google Scholar

Opis architektury biblioteki Tensorflow, https://developers.googleblog.com/2017/09/introducing-tensorflow-datasets.html, [26.06.2021].
  Google Scholar

D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, International Conference on Learning Representations (2015) 1–15.
  Google Scholar

T. Dozat, Incorporating Nesterov Momentum into Adam, International Conference on Learning Representations (2016) 1–4.
  Google Scholar

A. Lydia, S. Francis, Adagrad–an optimizer for stochastic gradient descent, International Journal of Information and Computing Science 6 (2019) 566–568.
  Google Scholar

M. D. Zeiler, Adadelta: an adaptive learning rate method, Computing Research Repository (2012).
  Google Scholar

M. Tokovarov, M. Kaczorowska, M. Miłosz, Development of Extensive Polish Handwritten Characters Database for Text Recognition Research, Advances in Science and Technology Research Journal 14 (2020) 30–38, https://doi.org/10.12913/22998624/122567.
DOI: https://doi.org/10.12913/22998624/122567   Google Scholar

E. Lukasik, M. Charytanowicz, M. Milosz, M. Tokovarov, M. Kaczorowska, D. Czerwinski, T. Zientarski, Recognition of handwritten Latin characters with diacritics using CNN, Bulletin of the Polish Academy of Sciences: Technical Sciences 69 (2021) 1–12, http://dx.doi.org/10.24425/bpasts.2020.136210.
  Google Scholar

E. Zitzler, Evolutionary algorithms for multiobjective optimization: Methods and applications, Ithaca: Shaker. 1999.
  Google Scholar

K. Kukuła, Metoda unitaryzacji zerowanej na tle wybranych metod normowania cech diagnostycznych, Acta Scientifica Academiae Ostroviensis 4 (1999) 5–31.
  Google Scholar

Download


Published
2021-12-30

Cited by

Wadas, M., & Smołka, J. (2021). Performance analysis of the TensorFlow library with different optimisation algorithms. Journal of Computer Sciences Institute, 21, 330–335. https://doi.org/10.35784/jcsi.2738

Authors

Maciej Wadas 
maciej.wadas@pollub.edu.pl
Politechnika Lubelska Wydział Elektrotechniki i Informatyki Poland

Authors

Jakub Smołka 

Lublin University of Technology Poland

Statistics

Abstract views: 237
PDF downloads: 229