STUDY ON DEEP LEARNING MODELS FOR THE CLASSIFICATION OF VR SICKNESS LEVELS
Haechan NA
lye7009@koreatech.ac.krKorea University of Technology and Education (Korea, Republic of)
Yoon Sang KIM
Korea University of Technology and Education, Institute for Bioengineering Application Technology, Department of Computer Science and Engineering, BioComputing Lab (Korea, Republic of)
https://orcid.org/0000-0002-0416-7938
Abstract
Virtual Reality (VR) sickness is often accompanied by symptoms such as nausea and dizziness, and a prominent theory explaining this phenomenon is the sensory conflict theory. Recently, studies have used Deep Learning to classify VR sickness levels; however, there is a paucity of research on Deep Learning models that utilize both visual information and motion data based on sensory conflict theory. In this paper, the authors propose a parallel merging of a Deep Learning model (4bay) to classify the level of VR sickness by utilizing the user's motion data (HMD, controller data) and visual data (rendered image, depth image) based on sensory conflict theory. The proposed model consists of a visual processing module, a motion processing module, and an FC-based VR sickness level classification module. The performance of the proposed model was compared with that of the developed models at the time of design. As a result of the comparison, it was confirmed that the proposed model performed better than the single model and the merged (2bay) model in classifying the user's VR sickness level.
Supporting Agencies
Keywords:
VR sickness, Cyber sickness, Deep Learning, LSTM, ResNet, Sensory Conflict TheoryReferences
Du, M., Cui, H., Wang, Y., & Duh, H. B. L. (2021). Learning from deep stereoscopic attention for simulator sickness prediction. IEEE Transactions on Visualization and Computer Graphics, 29(2), 1415-1423. https://doi.org/10.1109/TVCG.2021.3115901
DOI: https://doi.org/10.1109/TVCG.2021.3115901
Google Scholar
Falkowicz, K., & Kulisz, M. (2024). Prediction of buckling behaviour of composite plate element using artificial neural networks. Advances in Science and Technology. Research Journal, 18(1). https://doi.org/10.12913/22998624/177399
DOI: https://doi.org/10.12913/22998624/177399
Google Scholar
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual Learning for image recognition. IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770-778). IEEE. https://doi.org/10.1109/CVPR.2016.90
DOI: https://doi.org/10.1109/CVPR.2016.90
Google Scholar
Jeong, D., Paik, S., Noh, Y., & Han, K. (2023). MAC: multimodal, attention-based cybersickness prediction modeling in virtual reality. Virtual Reality, 27(3), 2315-2330. https://doi.org/10.1007/s10055-023-00804-0
DOI: https://doi.org/10.1007/s10055-023-00804-0
Google Scholar
Karpiński, R., Krakowski, P., Jonak, J., Machrowska, A., & Maciejewski, M. (2023). Comparison of selected classification methods based on Machine Learning as a diagnostic tool for knee joint cartilage damage based on generated vibroacoustic processes. Applied Computer Science, 19(4), 136-150. https://doi.org/10.35784/acs-2023-40
DOI: https://doi.org/10.35784/acs-2023-40
Google Scholar
Keshavarz, B., Peck, K., Rezaei, S., & Taati, B. (2022). Detecting and predicting visually induced motion sickness with physiological measures in combination with Machine Learning techniques. International Journal of Psychophysiology, 176, 14-26. https://doi.org/10.1016/j.ijpsycho.2022.03.006
DOI: https://doi.org/10.1016/j.ijpsycho.2022.03.006
Google Scholar
Kulisz, M., Kujawska, J., Cioch, M., Cel, W., & Pizoń, J. (2024). Comparative analysis of Machine Learning methods for predicting energy recovery from waste. Applied Sciences, 14(7), 2997. https://doi.org/10.3390/app14072997
DOI: https://doi.org/10.3390/app14072997
Google Scholar
Kundu, R. K., Islam, R., Quarles, J., & Hoque, K. A. (2023). LiteVR: Interpretable and lightweight cybersickness detection using explainable AI. 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR) (pp. 609-619). IEEE. https://doi.org/10.1109/VR55154.2023.00076
DOI: https://doi.org/10.1109/VR55154.2023.00076
Google Scholar
LaViola, Jr, J. J. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin, 32(1), 47-56. https://doi.org/10.1145/333329.333344
DOI: https://doi.org/10.1145/333329.333344
Google Scholar
Lim, H. K., Ji, K., Woo, Y. S., Han, D. U., Lee, D. H., Nam, S. G., & Jang, K. M. (2021). Test-retest reliability of the virtual reality sickness evaluation using electroencephalography (EEG). Neuroscience Letters, 743, 135589. https://doi.org/10.1016/j.neulet.2020.135589
DOI: https://doi.org/10.1016/j.neulet.2020.135589
Google Scholar
Monteiro, D., Liang, H. N., Tang, X., & Irani, P. (2021). Using trajectory compression rate to predict changes in cybersickness in virtual reality games. 2021 IEEE international symposium on mixed and augmented reality (ISMAR), (pp. 138-146). IEEE. https://doi.org/10.1109/ISMAR52148.2021.00028
DOI: https://doi.org/10.1109/ISMAR52148.2021.00028
Google Scholar
Ng, A. K. T., Chan, L. K. Y., & Lau, H. Y. K. (2020). A study of cybersickness and sensory conflict theory using a motion-coupled virtual reality system. Displays, 61, 101922. https://doi.org/10.1016/j.displa.2019.08.004
DOI: https://doi.org/10.1016/j.displa.2019.08.004
Google Scholar
Shimada, S., Ikei, Y., Nishiuchi, N., & Yem, V. (2023a). Study of cybersickness prediction in real time using eye tracking data. 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 871-872). IEEE. https://doi.org/10.1109/VRW58643.2023.00278
DOI: https://doi.org/10.1109/VRW58643.2023.00278
Google Scholar
Shimada, S., Pannattee, P., Ikei, Y., Nishiuchi, N., & Yem, V. (2023b). High-frequency cybersickness prediction using Deep Learning techniques with eye-related indices. IEEE Access, 11, 95825-95839. https://doi.org/10.1109/ACCESS.2023.3312216
DOI: https://doi.org/10.1109/ACCESS.2023.3312216
Google Scholar
Shodipe, O. E., & Allison, R. S. (2023). Modelling the relationship between the objective measures of car sickness. 2023 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 570-575). IEEE. https://doi.org/10.1109/CCECE58730.2023.10289000
DOI: https://doi.org/10.1109/CCECE58730.2023.10289000
Google Scholar
Wang, J., Liang, H. N., Monteiro, D. V., Xu, W., Chen, H., & Chen, Q. (2020). Real-time detection of simulator sickness in virtual reality games based on players' psychophysiological data during gameplay. 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (pp. 247-248). IEEE. https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00071
DOI: https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00071
Google Scholar
Wen, E., Gupta, C., Sasikumar, P., Billinghurst, M., Wilmott, J., Skow, E., Dey. A., & Nanayakkara, S. (2024). VR. net: A real-world dataset for virtual reality motion sickness research. IEEE Transactions on Visualization and Computer Graphics, 30(5), 2330-2336. https://doi.org/10.1109/TVCG.2024.3372044
DOI: https://doi.org/10.1109/TVCG.2024.3372044
Google Scholar
Yang, A. H. X., Kasabov, N., & Cakmak, Y. O. (2022). Machine Learning methods for the study of cybersickness: A systematic review. Brain Informatics, 9(1), 24. https://doi.org/10.1186/s40708-022-00172-6
DOI: https://doi.org/10.1186/s40708-022-00172-6
Google Scholar
Younis, M. C. (2024). Prediction of patient’s willingness for treatment of mental illness using Machine Learning approaches. Applied Computer Science, 20(2), 175-193. https://doi.org/10.35784/acs-2024-23
DOI: https://doi.org/10.35784/acs-2024-23
Google Scholar
Zhao, J., Tran, K. T., Chalmers, A., Hoh, W. K., Yao, R., Dey, A., Wilmott, J., Lin, J., Billinghurst, M., Lindeman, & Rhee, T. (2023). Deep Learning-based simulator sickness estimation from 3D motion. 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 39-48). IEEE. https://doi.org/10.1109/ISMAR59233.2023.00018
DOI: https://doi.org/10.1109/ISMAR59233.2023.00018
Google Scholar
Authors
Haechan NAlye7009@koreatech.ac.kr
Korea University of Technology and Education Korea, Republic of
Authors
Yoon Sang KIMKorea University of Technology and Education, Institute for Bioengineering Application Technology, Department of Computer Science and Engineering, BioComputing Lab Korea, Republic of
https://orcid.org/0000-0002-0416-7938
Statistics
Abstract views: 91PDF downloads: 36
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Most read articles by the same author(s)
- Hae Chan Na, Yoon Sang Kim, A STUDY ON AN AR-BASED CIRCUIT PRACTICE , Applied Computer Science: Vol. 20 No. 1 (2024)
Similar Articles
- Mariano LARIOS, Perfecto M. QUINTERO-FLORES , Mario ANZURES-GARCÍA , Miguel CAMACHO-HERNANDEZ , APPLICATION OF THE REAL-TIME FAN SCHEDULING IN THE EXPLORATION-EXPLOITATION TO OPTIMIZE MINIMUM FUNCTIONS OBJECTIVES , Applied Computer Science: Vol. 19 No. 2 (2023)
- Mouna TARIK, Ayoub MNIAI, Khalid JEBARI, HYBRID FEATURE SELECTION AND SUPPORT VECTOR MACHINE FRAMEWORK FOR PREDICTING MAINTENANCE FAILURES , Applied Computer Science: Vol. 19 No. 2 (2023)
- Tilla IZSÁK, László MARÁK, Mihály ORMOS, EVALUATION OF SUPPORT VECTOR MACHINE BASED STOCK PRICE PREDICTION , Applied Computer Science: Vol. 19 No. 3 (2023)
- Robert KARPIŃSKI, Przemysław KRAKOWSKI, Józef JONAK, Anna MACHROWSKA, Marcin MACIEJEWSKI, COMPARISON OF SELECTED CLASSIFICATION METHODS BASED ON MACHINE LEARNING AS A DIAGNOSTIC TOOL FOR KNEE JOINT CARTILAGE DAMAGE BASED ON GENERATED VIBROACOUSTIC PROCESSES , Applied Computer Science: Vol. 19 No. 4 (2023)
- Eduardo Sánchez-García, Javier Martínez-Falcó, Bartolomé Marco-Lajara, Jolanta Słoniec, ANALYZING THE ROLE OF COMPUTER SCIENCE IN SHAPING MODERN ECONOMIC AND MANAGEMENT PRACTICES. BIBLIOMETRIC ANALYSIS , Applied Computer Science: Vol. 20 No. 1 (2024)
- Monika KULISZ, Aigerim DUISENBEKOVA, Justyna KUJAWSKA, Danira KALDYBAYEVA, Bibigul ISSAYEVA, Piotr LICHOGRAJ, Wojciech CEL, IMPLICATIONS OF NEURAL NETWORK AS A DECISION-MAKING TOOL IN MANAGING KAZAKHSTAN’S AGRICULTURAL ECONOMY , Applied Computer Science: Vol. 19 No. 4 (2023)
- Hae Chan Na, Yoon Sang Kim, A STUDY ON AN AR-BASED CIRCUIT PRACTICE , Applied Computer Science: Vol. 20 No. 1 (2024)
- Ghania ZIDANI, Djalal DJARAH, Abdslam BENMAKHLOUF, Laid KHETTACHE, OPTIMIZING PEDESTRIAN TRACKING FOR ROBUST PERCEPTION WITH YOLOv8 AND DEEPSORT , Applied Computer Science: Vol. 20 No. 1 (2024)
- Janusz MLECZKO, Paweł BOBIŃSKI, PRODUCTION PLANNING IN CONDITIONS OF MASS CUSTOMIZATION BASED ON THEORY OF CONSTRAINTS , Applied Computer Science: Vol. 13 No. 4 (2017)
- Siti ROHAJAWATI, Hutanti SETYODEWI, Ferryansyah Muji Agustian TRESNANTO, Debora MARIANTHI, Maruli Tua Baja SIHOTANG , KNOWLEDGE MANAGEMENT APPROACH IN COMPARATIVE STUDY OF AIR POLLUTION PREDICTION MODEL , Applied Computer Science: Vol. 20 No. 1 (2024)
You may also start an advanced similarity search for this article.
