STUDY ON DEEP LEARNING MODELS FOR THE CLASSIFICATION OF VR SICKNESS LEVELS
Haechan NA
lye7009@koreatech.ac.krKorea University of Technology and Education (Korea, Republic of)
Yoon Sang KIM
Korea University of Technology and Education, Institute for Bioengineering Application Technology, Department of Computer Science and Engineering, BioComputing Lab (Korea, Republic of)
https://orcid.org/0000-0002-0416-7938
Abstract
Virtual Reality (VR) sickness is often accompanied by symptoms such as nausea and dizziness, and a prominent theory explaining this phenomenon is the sensory conflict theory. Recently, studies have used Deep Learning to classify VR sickness levels; however, there is a paucity of research on Deep Learning models that utilize both visual information and motion data based on sensory conflict theory. In this paper, the authors propose a parallel merging of a Deep Learning model (4bay) to classify the level of VR sickness by utilizing the user's motion data (HMD, controller data) and visual data (rendered image, depth image) based on sensory conflict theory. The proposed model consists of a visual processing module, a motion processing module, and an FC-based VR sickness level classification module. The performance of the proposed model was compared with that of the developed models at the time of design. As a result of the comparison, it was confirmed that the proposed model performed better than the single model and the merged (2bay) model in classifying the user's VR sickness level.
Supporting Agencies
Keywords:
VR sickness, Cyber sickness, Deep Learning, LSTM, ResNet, Sensory Conflict TheoryReferences
Du, M., Cui, H., Wang, Y., & Duh, H. B. L. (2021). Learning from deep stereoscopic attention for simulator sickness prediction. IEEE Transactions on Visualization and Computer Graphics, 29(2), 1415-1423. https://doi.org/10.1109/TVCG.2021.3115901
DOI: https://doi.org/10.1109/TVCG.2021.3115901
Google Scholar
Falkowicz, K., & Kulisz, M. (2024). Prediction of buckling behaviour of composite plate element using artificial neural networks. Advances in Science and Technology. Research Journal, 18(1). https://doi.org/10.12913/22998624/177399
DOI: https://doi.org/10.12913/22998624/177399
Google Scholar
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual Learning for image recognition. IEEE conference on computer vision and pattern recognition (CVPR) (pp. 770-778). IEEE. https://doi.org/10.1109/CVPR.2016.90
DOI: https://doi.org/10.1109/CVPR.2016.90
Google Scholar
Jeong, D., Paik, S., Noh, Y., & Han, K. (2023). MAC: multimodal, attention-based cybersickness prediction modeling in virtual reality. Virtual Reality, 27(3), 2315-2330. https://doi.org/10.1007/s10055-023-00804-0
DOI: https://doi.org/10.1007/s10055-023-00804-0
Google Scholar
Karpiński, R., Krakowski, P., Jonak, J., Machrowska, A., & Maciejewski, M. (2023). Comparison of selected classification methods based on Machine Learning as a diagnostic tool for knee joint cartilage damage based on generated vibroacoustic processes. Applied Computer Science, 19(4), 136-150. https://doi.org/10.35784/acs-2023-40
DOI: https://doi.org/10.35784/acs-2023-40
Google Scholar
Keshavarz, B., Peck, K., Rezaei, S., & Taati, B. (2022). Detecting and predicting visually induced motion sickness with physiological measures in combination with Machine Learning techniques. International Journal of Psychophysiology, 176, 14-26. https://doi.org/10.1016/j.ijpsycho.2022.03.006
DOI: https://doi.org/10.1016/j.ijpsycho.2022.03.006
Google Scholar
Kulisz, M., Kujawska, J., Cioch, M., Cel, W., & Pizoń, J. (2024). Comparative analysis of Machine Learning methods for predicting energy recovery from waste. Applied Sciences, 14(7), 2997. https://doi.org/10.3390/app14072997
DOI: https://doi.org/10.3390/app14072997
Google Scholar
Kundu, R. K., Islam, R., Quarles, J., & Hoque, K. A. (2023). LiteVR: Interpretable and lightweight cybersickness detection using explainable AI. 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR) (pp. 609-619). IEEE. https://doi.org/10.1109/VR55154.2023.00076
DOI: https://doi.org/10.1109/VR55154.2023.00076
Google Scholar
LaViola, Jr, J. J. (2000). A discussion of cybersickness in virtual environments. ACM Sigchi Bulletin, 32(1), 47-56. https://doi.org/10.1145/333329.333344
DOI: https://doi.org/10.1145/333329.333344
Google Scholar
Lim, H. K., Ji, K., Woo, Y. S., Han, D. U., Lee, D. H., Nam, S. G., & Jang, K. M. (2021). Test-retest reliability of the virtual reality sickness evaluation using electroencephalography (EEG). Neuroscience Letters, 743, 135589. https://doi.org/10.1016/j.neulet.2020.135589
DOI: https://doi.org/10.1016/j.neulet.2020.135589
Google Scholar
Monteiro, D., Liang, H. N., Tang, X., & Irani, P. (2021). Using trajectory compression rate to predict changes in cybersickness in virtual reality games. 2021 IEEE international symposium on mixed and augmented reality (ISMAR), (pp. 138-146). IEEE. https://doi.org/10.1109/ISMAR52148.2021.00028
DOI: https://doi.org/10.1109/ISMAR52148.2021.00028
Google Scholar
Ng, A. K. T., Chan, L. K. Y., & Lau, H. Y. K. (2020). A study of cybersickness and sensory conflict theory using a motion-coupled virtual reality system. Displays, 61, 101922. https://doi.org/10.1016/j.displa.2019.08.004
DOI: https://doi.org/10.1016/j.displa.2019.08.004
Google Scholar
Shimada, S., Ikei, Y., Nishiuchi, N., & Yem, V. (2023a). Study of cybersickness prediction in real time using eye tracking data. 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) (pp. 871-872). IEEE. https://doi.org/10.1109/VRW58643.2023.00278
DOI: https://doi.org/10.1109/VRW58643.2023.00278
Google Scholar
Shimada, S., Pannattee, P., Ikei, Y., Nishiuchi, N., & Yem, V. (2023b). High-frequency cybersickness prediction using Deep Learning techniques with eye-related indices. IEEE Access, 11, 95825-95839. https://doi.org/10.1109/ACCESS.2023.3312216
DOI: https://doi.org/10.1109/ACCESS.2023.3312216
Google Scholar
Shodipe, O. E., & Allison, R. S. (2023). Modelling the relationship between the objective measures of car sickness. 2023 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 570-575). IEEE. https://doi.org/10.1109/CCECE58730.2023.10289000
DOI: https://doi.org/10.1109/CCECE58730.2023.10289000
Google Scholar
Wang, J., Liang, H. N., Monteiro, D. V., Xu, W., Chen, H., & Chen, Q. (2020). Real-time detection of simulator sickness in virtual reality games based on players' psychophysiological data during gameplay. 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) (pp. 247-248). IEEE. https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00071
DOI: https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00071
Google Scholar
Wen, E., Gupta, C., Sasikumar, P., Billinghurst, M., Wilmott, J., Skow, E., Dey. A., & Nanayakkara, S. (2024). VR. net: A real-world dataset for virtual reality motion sickness research. IEEE Transactions on Visualization and Computer Graphics, 30(5), 2330-2336. https://doi.org/10.1109/TVCG.2024.3372044
DOI: https://doi.org/10.1109/TVCG.2024.3372044
Google Scholar
Yang, A. H. X., Kasabov, N., & Cakmak, Y. O. (2022). Machine Learning methods for the study of cybersickness: A systematic review. Brain Informatics, 9(1), 24. https://doi.org/10.1186/s40708-022-00172-6
DOI: https://doi.org/10.1186/s40708-022-00172-6
Google Scholar
Younis, M. C. (2024). Prediction of patient’s willingness for treatment of mental illness using Machine Learning approaches. Applied Computer Science, 20(2), 175-193. https://doi.org/10.35784/acs-2024-23
DOI: https://doi.org/10.35784/acs-2024-23
Google Scholar
Zhao, J., Tran, K. T., Chalmers, A., Hoh, W. K., Yao, R., Dey, A., Wilmott, J., Lin, J., Billinghurst, M., Lindeman, & Rhee, T. (2023). Deep Learning-based simulator sickness estimation from 3D motion. 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (pp. 39-48). IEEE. https://doi.org/10.1109/ISMAR59233.2023.00018
DOI: https://doi.org/10.1109/ISMAR59233.2023.00018
Google Scholar
Authors
Haechan NAlye7009@koreatech.ac.kr
Korea University of Technology and Education Korea, Republic of
Authors
Yoon Sang KIMKorea University of Technology and Education, Institute for Bioengineering Application Technology, Department of Computer Science and Engineering, BioComputing Lab Korea, Republic of
https://orcid.org/0000-0002-0416-7938
Statistics
Abstract views: 91PDF downloads: 36
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles published in Applied Computer Science are open-access and distributed under the terms of the Creative Commons Attribution 4.0 International License.
Most read articles by the same author(s)
- Hae Chan Na, Yoon Sang Kim, A STUDY ON AN AR-BASED CIRCUIT PRACTICE , Applied Computer Science: Vol. 20 No. 1 (2024)
Similar Articles
- Lubna RIYAZ, Muheet Ahmed BUTT, Majid ZAMAN, IMPROVING CORONARY HEART DISEASE PREDICTION BY OUTLIER ELIMINATION , Applied Computer Science: Vol. 18 No. 1 (2022)
- Nouhaila BOUALOULOU, Taoufiq BELHOUSSINE DRISSI, Benayad NSIRI, CNN AND LSTM FOR THE CLASSIFICATION OF PARKINSON'S DISEASE BASED ON THE GTCC AND MFCC , Applied Computer Science: Vol. 19 No. 2 (2023)
- Amina ALYAMANI, Oleh YASNIY, CLASSIFICATION OF EEG SIGNAL BY METHODS OF MACHINE LEARNING , Applied Computer Science: Vol. 16 No. 4 (2020)
- Fernando Andrés CEVALLOS SALAS, DIGITAL NEWS CLASSIFICATION AND PUNCTUACTION USING MACHINE LEARNING AND TEXT MINING TECHNIQUES , Applied Computer Science: Vol. 20 No. 2 (2024)
- Baigo HAMUNA, Sri PUJIYATI, Jonson Lumban GAOL, Totok HESTIRIANOTO, CLASSIFICATION AND PREDICTION OF BENTHIC HABITAT FROM SCIENTIFIC ECHOSOUNDER DATA: APPLICATION OF MACHINE LEARNING ALGORITHMS , Applied Computer Science: Vol. 20 No. 4 (2024)
- Bilal OWAIDAT, EXPLORING THE ACCURACY AND RELIABILITY OF MACHINE LEARNING APPROACHES FOR STUDENT PERFORMANCE , Applied Computer Science: Vol. 20 No. 3 (2024)
- Roman GALAGAN, Serhiy ANDREIEV, Nataliia STELMAKH, Yaroslava RAFALSKA, Andrii MOMOT, AUTOMATION OF POLYCYSTIC OVARY SYNDROME DIAGNOSTICS THROUGH MACHINE LEARNING ALGORITHMS IN ULTRASOUND IMAGING , Applied Computer Science: Vol. 20 No. 2 (2024)
- Sri INDRA MAIYANTI, Anita DESIANI, Syafrina LAMIN, P PUSPITAHATI, Muhammad ARHAMI, Nuni GOFAR, Destika CAHYANA, ROTATION-GAMMA CORRECTION AUGMENTATION ON CNN-DENSE BLOCK FOR SOIL IMAGE CLASSIFICATION , Applied Computer Science: Vol. 19 No. 3 (2023)
- Elmehdi BENMALEK, Jamal EL MHAMDI, Abdelilah JILBAB, Atman JBARI, A COUGH-BASED COVID-19 DETECTION SYSTEM USING PCA AND MACHINE LEARNING CLASSIFIERS , Applied Computer Science: Vol. 18 No. 4 (2022)
- Erizal ERIZAL, Mohammad DIQI, PERFORMANCE EVALUATION OF STOCK PREDICTION MODELS USING EMAGRU , Applied Computer Science: Vol. 19 No. 3 (2023)
You may also start an advanced similarity search for this article.
