Performance analysis of machine learning libraries


Abstract

The paper presents results of performance analysis of machine learning libraries. The research was based on ML.NET and TensorFlow tools. The analysis was based on a comparison of running time of the libraries, during detection of objects on sets of images, using hardware with different parameters. The library, consuming fewer hardware resources, turned out to be TensorFlow. The choice of hardware platform and the possibility of using graphic cores, affecting the increase in computational efficiency, turned out to be not without significance.


Keywords

machine learning; performance; ML.NET; TensorFlow

Leading business in the age of AI, https://news.microsoft.com/europe/features/leaders-look-to-embrace-ai-and-high-growth-companies-are-seeing-the-benefits, [29.11.2020].

Przewodnik po strukturze ML.NET, https://docs.microsoft.com/pl-pl/dotnet/machine-learning/how-does-mldotnet-work, [30.11.2020].

M. N. Gevorkyan, A. V. Demidova, T. S. Demidova, A. Sobolev, Review and comparative analysis of machine learning libraries for machine learning, Discrete And Continuous Models And Applied Computational Science 27 (2019) 305-315, http://dx.doi.org/10.22363/2658-4670-2019-27-4-305-315.

A. Géron, Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow, O'Reilly, 2019.

E. Brill, M. Banko, Scaling to very large corpora for natural language disambiguation, Proceedings of 39th Annual Meeting on Association for Computational Linguistics (2001) 26-33, https://doi.org/10.3115/1073012.1073017.

V. Shankar, R. Roelofs, H. Mania, A. Fang, B. Recht, L. Schmidt, Evaluating Machine Accuracy on ImageNet, 37th International Conference on Machine Learning (2020) 8634-8644.

E. Zuccarelli, Using machine learning to predict car accidents, https://towardsdatascience.com/using-machine-learning-to-predict-car-accidents-44664c79c942, [15.06.2021].

M. Hartley, T. S. G. Olsson, dtoolAI: Reproducibility for Deep Learning, Patterns 1(5) (2020) 100099, https://doi.org/10.1016/j.patter.2020.100073.

C. Deng, X. Ji, C. Rainey, J. Zhang, W. Lu, Integrating Machine Learning with Human Knowledge, iScience 23(11) (2020) 101656, https://doi.org/10.1016/j.isci.2020.101656.

Optimize TensorFlow performance using the Profiler, https://www.tensorflow.org/guide/profiler, [15.06.2021].

G. Nguyen, S. Dlugolinsky, M. Bobák, V. Tran, Á. García, I. Heredia, P. Malík, L. Hluchý, Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey, Artificial Intelligence Review 52 (2019) 77-124, https://doi.org/10.1007/s10462-018-09679-z.

F. Florencio, E. D. M. Ordonez, T. V. Silva, M. C. Júnior, Performance Analysis of Deep Learning Libraries: TensorFlow and PyTorch, Journal of Computer Science 15 (2019) 785-799, http://dx.doi.org/10.3844/jcssp.2019.785.799.

Z. Ahmed, S. Amizadeh, M. Bilenko, R. Carr, W.-S. Chin, Y. Dekel, X. Dupre, V. Eksarevskiy, E. Erhardt, C. Eseanu, S. Filipi, T. Finley, A. Goswami, M. Hoover, S. Inglis, M. Interlandi, S. Katzenber, Machine Learning at Microsoft with ML.NET, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2019) 2448-2458, https://doi.org/10.1145/3292500.3330667.

T. Jin, G.-T. Bercea, T. D. Le, T. Chen, G. Su, H. Imai, Y. Negishi, A. Leu, K. O'Brien, K. Kawachiya, A. E. Eichenberger, Compiling ONNX Neural Network Models Using MLIR (2020), https://arxiv.org/abs/2008.08272.

J. Redmon, YOLO: Real-Time Object Detection, https://pjreddie.com/darknet/yolov2, [10.06.2021].

J. Redmon, A. Farhadi, YOLO9000:Better, Faster, Stronger (2016), https://arxiv.org/abs/1612.08242.

W. Fang, L. Wang, P. Ren, Tinier-YOLO: A Real-Time Object Detection, IEEE Access 8 (2019) 1935-1944, https://doi.org/10.1109/ACCESS.2019.2961959.

Download

Published : 2021-09-30


Kędziora, E. J., & Maksim, G. K. (2021). Performance analysis of machine learning libraries. Journal of Computer Sciences Institute, 20, 230-236. https://doi.org/10.35784/jcsi.2693

Ewa Justyna Kędziora 
Grzegorz Krzysztof Maksim  grzegorz.maksim@pollub.edu.pl