СROSS PLATFORM TOOLS FOR MODELING AND RECOGNITION OF THE FINGERSPELLING ALPHABET OF GESTURE LANGUAGE
Serhii Kondratiuk
Taras Shevchenko National University of Kyiv (Ukraine)
http://orcid.org/0000-0002-5048-2576
Iurii Krak
krak@univ.kiev.ua1Taras Shevchenko National University of Kyiv, 2Glushkov Institute of Cybernetics of NAS of Ukraine (Ukraine)
http://orcid.org/0000-0002-8043-0785
Waldemar Wójcik
Lublin Univeristy of Technology, Institute of Electronics and Computer Science (Poland)
http://orcid.org/0000-0002-0843-8053
Abstract
A solution for the problems of the finger spelling alphabet of gesture language modelling and recognition based on cross-platform technologies is proposed. Modelling and recognition performance can be flexible and adjusted, based on the hardware it operates or based on the availability of an internet connection. The proposed approach tunes the complexity of the 3D hand model based on the CPU type, amount of available memory and internet connection speed. Sign recognition is also performed using cross-platform technologies and the tradeoff in model size and performance can be adjusted. the methods of convolutional neural networks are used as tools for gestures of alphabet recognition. For the gesture recognition experiment, a dataset of 50,000 images was collected, with 50 different hands recorded, with almost 1,000 images per each person. The experimental researches demonstrated the effectiveness of proposed approaches.
Keywords:
cross platform, sign language, fingerspelling alphabet, 3D modeling, Convolutional Neural NetworksReferences
Apple Touchless Gesture System for iDevices http://www.patentlyapple.com/patently-apple/2014/12/apple-invents-a-highly-advanced-air-gesturing-system-for-future-idevices-and-beyond.html (available 15.05.2019).
Google Scholar
ASL Sign language dictionary http://www.signasl.org/sign/model (available 15.05.2019).
Google Scholar
Howard A.G., Wang W.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications https://arxiv.org/pdf/1704.04861.pdf (available 15.05.2019).
Google Scholar
Khan R.Z., Ibraheem N.A., Meghanathan N., et al.: Comparative study of hand gesture recognition system. SIPM, FCST, ITCA, WSE, ACSIT, CS & IT 06/2012, 203–213.
Google Scholar
Krak I., Kondratiuk S.: Cross-platform software for the development of sign communication system: Dactyl language modelling, Proceedings of the 12th International Scientific and Technical Conference on Computer Sciences and Information Technologies, CSIT 1/2017, 167–170 [DOI: 10.1109/STC-CSIT.2017.8098760].
Google Scholar
Krizhevsky I. Sutskever, Hinton G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 2012, 1097–1105.
Google Scholar
Kryvonos I.G., Krak I.V., Barchukova Y., Trotsenko B.A.: Human hand motion parametrization for dactylemes modeling. Journal of Automation and Information Sciences 43(12)/2011, 1–11.
Google Scholar
Kryvonos I.G., Krak I.V., Barmak O.V., Shkilniuk D.V.: Construction and identification of elements of sign communication. Cybernetics and Systems Analysis 49(2)/2013, 163–172.
Google Scholar
Kryvonos I.G., Krak I.V.: Modeling human hand movements, facial expressions, and articulation to synthesize and visualize gesture information. Cybernetics and Systems Analysis 47(4)/2011, 501–505.
Google Scholar
Mell P., Grance T.: The NIST Definition of Cloud Computing (Technical report). National Institute of Standards and Technology: U.S. Department of Commerce, 2011 [DOI:10.6028/NIST.SP.800-145].
Google Scholar
Neff M., Kipp M., Albrecht I., Seidel H.P.: Gesture Modeling and Animation by Imitation. MPI–I 4/2006.
Google Scholar
Ong E.I., et al. : Sign language recognition using sequential pattern trees. Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012, 2200–2207.
Google Scholar
Raheja J.: Android based portable hand sign recognition system. 2015 [DOI: 10.15579/gcsr.vol3.ch1].
Google Scholar
Shapiro A., Chu D., Allen B., Faloutsos P.: Dynamic Controller Toolkit, 2005 http://www.arishapiro.com/Sandbox07_DynamicToolkit.pdf (available 15.05.2019).
Google Scholar
Smith J., Navi R.: The Architecture of Virtual Machines. Computer. IEEE Computer Society 38(5)/2005, 32–38.
Google Scholar
Tensorflow framework documentation https://www.tensorflow.org/api/ (available 15.05.2019).
Google Scholar
The Linux Information Project, Cross-platform Definition.
Google Scholar
Unity3D framework https://unity3d.com/ (available 15.05.2019).
Google Scholar
YAML – The Official YAML Web Site http://yaml.org/ (available 15.05.2019).
Google Scholar
Authors
Serhii KondratiukTaras Shevchenko National University of Kyiv Ukraine
http://orcid.org/0000-0002-5048-2576
Authors
Iurii Krakkrak@univ.kiev.ua
1Taras Shevchenko National University of Kyiv, 2Glushkov Institute of Cybernetics of NAS of Ukraine Ukraine
http://orcid.org/0000-0002-8043-0785
Authors
Waldemar WójcikLublin Univeristy of Technology, Institute of Electronics and Computer Science Poland
http://orcid.org/0000-0002-0843-8053
Statistics
Abstract views: 334PDF downloads: 178
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Most read articles by the same author(s)
- Madina Bazarova, Waldemar Wójcik, Gulnaz Zhomartkyzy, Saule Kumargazhanova, Galina Popova , KNOWLEDGE TRANSFER AS ONE OF THE FACTORS OF INCREASING UNIVERSITY COMPETITIVENESS , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 9 No. 3 (2019)
- Vasyl Kukharchuk, Waldemar Wójcik, Sergii Pavlov, Samoil Katsyv, Volodymyr Holodiuk, Oleksandr Reyda, Ainur Kozbakova, Gaukhar Borankulova , FEATURES OF THE ANGULAR SPEED DYNAMIC MEASUREMENTS WITH THE USE OF AN ENCODER , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 12 No. 3 (2022)
- Liudmyla Shkilniak, Waldemar Wójcik, Sergii Pavlov, Oleg Vlasenko, Tetiana Kanishyna, Irina Khomyuk, Oleh Bezverkhyi, Sofia Dembitska, Orken Mamyrbayev, Aigul Iskakova, EXPERT FUZZY SYSTEMS FOR EVALUATION OF INTENSITY OF REACTIVE EDEMA OF SOFT TISSUES IN PATIENTS WITH DIABETES , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 12 No. 3 (2022)
- Waldemar Wójcik, Maksat Kalimoldayev, Yedilkhan Amirgaliyev, Murat Kunelbayev, Aliya Kalizhanova, Ainur Kozbakova, Timur Merembayev, EXERGY ANALYSIS OF DOUBLE-CIRCUIT FLAT SOLAR COLLECTOR WITH THERMOSYPHON CIRCULATION , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 9 No. 3 (2019)
- Waldemar Wójcik, Aliya Kalizhanova, Gulzhan Kashaganova, Ainur Kozbakova, Zhalau Aitkulov, Zhassulan Orazbekov, RESEARCH OF PARAMETERS OF FIBER-OPTICAL MEASURING SYSTEMS , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 9 No. 2 (2019)
- Zbigniew Omiotek, Waldemar Wójcik, THE USE OF HELLWIG’S METHOD FOR DIMENSION REDUCTION IN FEATURE SPACE OF THYROID ULTRASOUND IMAGES , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 4 No. 3 (2014)
- Saltanat Adikanova, Waldemar Wójcik, Natalya Denissova, Yerzhan Malgazhdarov, Ainagul Kadyrova, DETERMINATION OF THE PROBABILITY FACTOR OF PARTICLES MOVEMENT IN A GAS-DISPERSED TURBULENT FLOW , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 9 No. 2 (2019)
- Ainur Orazayeva , Jamalbek Tussupov, Waldemar Wójcik, Sergii Pavlov, Gulzira Abdikerimova, Liudmyla Savytska, METHODS FOR DETECTING AND SELECTING AREAS ON TEXTURE BIOMEDICAL IMAGES OF BREAST CANCER , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 12 No. 2 (2022)
- Kuanysh Mussilimov, Akhmet Ibraev, Waldemar Wójcik, DEVELOPMENT OF WIND ENERGY COMPLEX AUTOMATION SYSTEM , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 9 No. 2 (2019)
- Waldemar Wójcik, Batyrbek Suleimenov, Gennadiy Shadrin, Mikhail Shadrin, Dmitriy Porubov, OPTIMAL CONTROL SYSTEM OF DIESEL AUTOMOTIVE ENGINEERING BY EXAMPLE OF OPEN PIT MOTOR TRANSPORT , Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska: Vol. 4 No. 1 (2014)
