COMPARISON OF OPTIMIZATION ALGORITHMS OF CONNECTIONIST TEMPORAL CLASSIFIER FOR SPEECH RECOGNITION SYSTEM

Yedilkhan Amirgaliyev; Kuanyshbay Kuanyshbay; Aisultan Shoiynbek

doi:10.35784/iapgos.234

Open full text

pdf

Published: Sep 26, 2019

DOI: https://doi.org/10.35784/iapgos.234

Issue Vol. 9 No. 3 (2019)

Articles

KNOWLEDGE TRANSFER AS ONE OF THE FACTORS OF INCREASING UNIVERSITY COMPETITIVENESS
Madina Bazarova, Waldemar Wójcik, Gulnaz Zhomartkyzy, Saule Kumargazhanova, Galina Popova

4-9
GENERATIONS IN BAYESIAN NETWORKS
Alexander Litvinenko, Natalya Litvinenko, Orken Mamyrbayev, Assem Shayakhmetova

10-13
MODEL OF ENGINEERING EDUCATION WITH THE USE OF THE COMPETENCE-PROJECT APPROACH, ONTOLOGICAL ENGINEERING AND SMART CONTRACTS OF KNOWLEDGE COMPONENTS
Bulat Kubekov, Leonid Bobrov, Anar Utegenova, Vitaly Naumenko, Raigul Alenova

14-17
ASSESSMENT OF THE DIAGNOSTIC VALUE OF THE METHOD OF COMPUTER OLFACTOMETRY
Oleg Avrunin, Yana Nosova, Sergii Zlepko, Ibrahim Younouss Abdelhamid , Nataliia Shushliapina

18-21
DIFFERENTIAL DIAGNOSTICS OF ASEPTIC AND SEPTIC LOOSENING OF THE CUP OF THE ENDOPROSTHESIS OF THE ARTIFICIAL HIP JOINT BY THE METHODS OF POLARISATION TOMOGRAPHY
Alexander G. Ushenko, Olexander Olar

22-25
THE ANALYSIS OF APPROACHES TO MEASUREMENT UNCERTAINTY EVALUATION FOR CALIBRATION
Sarsenbek Zhussupbekov, Svetlana Khan, Lida Ibrayeva

26-29
TOOL CONTROL THE CONCENTRATION OF CARBON DIOXIDE IN THE FLUE GAS BOILERS BASED ON THE OPTICAL ABSORPTION METHOD
Oleksandr Vasilevskyi, Ihor Dudatiev, Kostyantyn Ovchynnykov

30-34
EXERGY ANALYSIS OF DOUBLE-CIRCUIT FLAT SOLAR COLLECTOR WITH THERMOSYPHON CIRCULATION
Waldemar Wójcik, Maksat Kalimoldayev, Yedilkhan Amirgaliyev, Murat Kunelbayev, Aliya Kalizhanova, Ainur Kozbakova, Timur Merembayev

35-39
ANALYSIS OF SOIL ORGANIC MATTER TRANSFORMATION DYNAMICS MODELS
Liubov Shostak, Mykhailo Boiko, Olha Stepanchenko, Olena Kozhushko

40-45
METHOD OF DIAGNOSTICS OF FILLING MATERIAL STRENGTH BASED ON TIME SERIES
Salim Mustafin, Marat Arslanov, Abdikarim Zeinullin, Ekaterina Korobova

46-49
APPLICATION OF CLONAL SELECTION ALGORITHM FOR PID CONTROLLER SYNTHESIS OF MIMO SYSTEMS IN OIL AND GAS INDUSTRY
Olga Shiryayeva, Timur Samigulin

50-53
COMPARISON OF OPTIMIZATION ALGORITHMS OF CONNECTIONIST TEMPORAL CLASSIFIER FOR SPEECH RECOGNITION SYSTEM
Yedilkhan Amirgaliyev, Kuanyshbay Kuanyshbay, Aisultan Shoiynbek

54-57
SYNTHESIS OF A TRACKING CONTROL SYSTEM OVER THE FLOTATION PROCESS BASED ON LQR-ALGORITHM
Shamil Koshimbaey, Zhanar Lukmanova, Andrzej Smolarz, Shynggyskhan Auyelbek

58-61
THE METHOD OF DETECTING INHOMOGENEITIES AND DEFECTS IN MATERIALS USING SENSORS BASED ON THE FIBER BRAGG OPTIC STRUCTURES
Łukasz Zychowicz

62-65
COMPARISON OF THE INFLUENCE OF STANDARDIZATION AND NORMALIZATION OF DATA ON THE EFFECTIVENESS OF SPONGY TISSUE TEXTURE CLASSIFICATION
Róża Dzierżak

66-69
DIAGNOSTIC OF THE COMBUSTION PROCESS USING THE ANALYSIS OF CHANGES IN FLAME LUMINOSITY
Żaklin Grądz, Joanna Styczeń

70-73
AN OPTIMIZATION OF A DIGITAL CONTROLLER FOR A STOCHASTIC CONTROL SYSTEM
Igor Golinko, Volodymyr Drevetskiy

74-77
THE SYSTEM OF COUNTERACTION TO UNMANNED AERIAL VEHICLES
Nataliia Lishchyna, Valerii Lishchyna, Yuliia Povstiana, Andrii Yashchuk

78-81
AGRICULTURAL MANAGEMENT ON THE BASIS OF INFORMATION TECHNOLOGIES
Olena Sivakovska, Mykola Rudinets, Mykhailo Poteichuk

82-85
DYE PHOTOSENSITIZERS AND THEIR INFLUENCE ON DSSC EFFICIENCY: A REVIEW
Ewelina Krawczak

86-90

DOI

https://doi.org/10.35784/iapgos.234

Authors

Yedilkhan Amirgaliyev

amir_ed@mail.ru

1 Institute Information and Computational Technologies CS MES RK, Almaty, Kazakhstan, 2 Suleyman Demirel University, Almaty, Kazakhstan, Kazakhstan

http://orcid.org/0000-0002-6528-0619

Kuanyshbay Kuanyshbay

darkhan.kuanyshbay@sdu.edu.kz

1 Institute Information and Computational Technologies CS MES RK, Almaty, Kazakhstan, 2 Suleyman Demirel University, Almaty, Kazakhstan, Kazakhstan

http://orcid.org/0000-0001-5952-8609

Aisultan Shoiynbek

aisultan.shoiynbek@sdu.edu.kz

1 Institute Information and Computational Technologies CS MES RK, Almaty, Kazakhstan, 2 Suleyman Demirel University, Almaty, Kazakhstan, Kazakhstan

http://orcid.org/0000-0002-9328-8300

Abstract

This paper evaluates and compares the performances of three well-known optimization algorithms (Adagrad, Adam, Momentum) for faster training the neural network of CTC algorithm for speech recognition. For CTC algorithms recurrent neural network has been used, specifically Long-Short-Term memory. LSTM is effective and often used model. Data has been downloaded from VCTK corpus of Edinburgh University. The results of optimization algorithms have been evaluated by the Label error rate and CTC loss.

Keywords:

recurrent neural network, search methods, acoustic, systems modeling language

References

Amirgaliev Y., Hahn M., Mussabayev T.: The speech signal segmentation algorithm using pitch synchronous analysis. Journal Open Computer Science 7(1)/2017, 1–8. DOI: https://doi.org/10.1515/comp-2017-0001

Andrychowicz M., Denil M., Colmenarejo S.G., Hoffman M.W., Pfau D., Schaul T., Shillingford B., de Freitas N.: Learning to learn by gradient descent by gradient descent. 30th Conference on Neural Information Processing Systems NIPS 2016.

Bahdanau D., Cho K., Bengio Y.: Neural machine translation by jointly learning to align and translate. Proc. ICLR, 2015.

Bengio Y., Ducharme R., Vincent P., Jauvin C.: A Neural Probabilistic Language Model. Journal of Machine Learning Research 3/2003, 1137–1155.

Bottou L.: Large-Scale Machine Learning with Stochastic Gradient Descent. NEC Labs America, Princeton.

Duchi J., Hazan E., Singer Y.: Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research 12/2011, 2121–2159.

Gales M., Young S.: The Application of Hidden Markov Models in Speech Recognition. Foundations and Trends in Signal Processing 1(3)/2007, 195–304. DOI: https://doi.org/10.1561/2000000004

Graves A., Fernandez S., Gomez F., Schmidhuber J.: Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, 2006. DOI: https://doi.org/10.1145/1143844.1143891

Graves A., Jaitly N.: Towards End-to-End Speech Recognition with Recurrent Neural Networks. Proceedings of the 31st International Conference on Machine Learning 2014.

Kingma D.P., Ba J.: Adam: A Method For Stochastic Optimization. Proc. 3rd International Conference for Learning Representations. 2015 arXiv:1412.6980v9.

Loizou N., Richtarik P.: Momentum and Stochastic Momentum for Stochastic Gradient, Newton, Proximal Point and Subspace Descent Methods. 2017, arXiv:1712.09677v2

Mussabayev R.R., Amirgaliyev N., Tairova A.T., Mussabayev T.R., Koibagarov K.C.: The technology for the automatic formation of the personal digital voice pattern. Application of Information and Communication Technologies AICT 2016. DOI: https://doi.org/10.1109/ICAICT.2016.7991733

Schuster M., Paliwal K.K.: Bidirectional recurrent neural networks. Signal Processing. IEEE Transactions 45(11)/1997, 2673–2681. DOI: https://doi.org/10.1109/78.650093

Sutskever I., Vinyals O., Le Q.V.: Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems 2014, 3104–3112.

Wiseman S., Rush A.M.: Sequence-to-Sequence Learning as Beam-Search Optimization. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing 2016. DOI: https://doi.org/10.18653/v1/D16-1137

Yu D., Li J.: Recent Progresses in Deep Learning based Acoustic Models. Tencent AI Lab, Microsoft AI and Research, 2018.

Amirgaliyev, Y., Kuanyshbay, K., & Shoiynbek, A. (2019). COMPARISON OF OPTIMIZATION ALGORITHMS OF CONNECTIONIST TEMPORAL CLASSIFIER FOR SPEECH RECOGNITION SYSTEM . Informatyka, Automatyka, Pomiary W Gospodarce I Ochronie Środowiska, 9(3), 54–57. https://doi.org/10.35784/iapgos.234

COMPARISON OF OPTIMIZATION ALGORITHMS OF CONNECTIONIST TEMPORAL CLASSIFIER FOR SPEECH RECOGNITION SYSTEM

Issue Vol. 9 No. 3 (2019)

Archives

DOI

Authors

Abstract

Keywords:

References

License

Article Sidebar

Issue Vol. 9 No. 3 (2019)

Archives

Main Article Content

DOI

Authors

Abstract

Keywords:

References

Article Details

License