Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario

Main Article Content

Wen-Hsiang Lin
Wei-Kai Liou
Pao-Ju Chen
Sufen Chen

Abstract

Background: Stethoscopes and mannequins are basic tools for medical practitioners to diagnose 
various problems. Restricted by space, resources, and time, these tools and common video training 
are sometimes inaccessible. These obstacles may be solved by immersive virtual reality (VR) that 
improves interaction and blurs the line between virtual and reality. Therefore, this study combines 
virtual and real interaction technologies to investigate the impact of multi-sensory interaction on 
learning outcomes in chest auscultation.
Methods: This study integrated VIVE Tracker technology to provide synchronized tactile stimulation 
in VR. University students in medical-related disciplines participated in the study. Interviews 
were conducted to understand how students perceived the effectiveness and usefulness of the 
thoracic auscultation VR.
Conclusion: Most students expressed that tactile stimulation gave realistic sensations and 
effectively reinforced the immersive effect. Simultaneously, it increased their interest and 
motivation in learning. The integration of synchronized tactile stimulation is a novel approach 
that expands the realm of interactivity beyond handheld controller manipulation and creates a 
platform to simulate medical-related scenarios to learn complex knowledge
and skills.

Article Details

How to Cite
Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario. (2024). International Journal of Nursing Education, 16(2), 45-50. https://doi.org/10.37506/5rbjt116
Section
Articles
Author Biographies

Wen-Hsiang Lin

Graduate Institute of Digital Learning and Education, National Taiwan University of Science and
Technology, Taiwan

Wei-Kai Liou

Empower Vocational Education Research Center, National Taiwan University of Science and Technology, Taiwan

Pao-Ju Chen

Department of Nursing, Asia Eastern University of Science and Technology, Taiwan

Sufen Chen

Graduate Institute of Digital Learning and Education, National Taiwan University of Science and
Technology, Taiwan Empower Vocational Education Research Center, National Taiwan University
of Science and Technology, Taiwan

How to Cite

Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario. (2024). International Journal of Nursing Education, 16(2), 45-50. https://doi.org/10.37506/5rbjt116

References

Hsieh MC, Lin YH. VR and AR applications in medical practice and education. The Journal of Nursing. 2017;64(6):12-18.https://10.6224/JN.000078

Matthews S, Wood K, Quevedo AJU, Jaimes N, Dubrowski A, Kapralos B, Alam F, Rojas D. Work-in-Progress–A preliminary eye tracking and HMD orientation comparison to determine focus on a cardiac auscultation training environment. In: Proceedings of the 7th International Conference of the Immersive Learning Research Network (ILRN); 2021; Eureka, CA, USA.https://10.23919/iLRN52045.2021.9459383

Yeh HF. Virtual reality skills training trends in nurse practitioner education. The Journal of Nursing. 2021;68(5):13-17. https://10.6224/JN.202110_68(5)

Chang YM, Lai CL. Exploring the experiences of nursing students in using immersive virtual reality to learn nursing skills. Nurse Education Today. 2021;97. https://10.1016/j.nedt.2020.104670

Hara CYN, Goes FDSN, Camargo CRAA, Fonseca LMM, Aredes NDA. Design and evaluation of a 3D serious game for communication learning in nursing education. Nurse Education Today. 2021;100. https://10.1016/j.nedt.2021.104846

Bagher MM, Bagher P, Bagher J, Bagher P, Klippel A. Move the object or move the user: The role of interaction techniques on embodied learning in VR. Frontiers in Virtual Reality. 2021;2. https://10.3389/frvir.2021.695312

Hannans JA, Nevins CM, Jordan K. See it, hear it, feel it: Embodying a patient experience through immersive virtual reality. Information and Learning Science. 2021;122(7-8):565-583. https://10.1108/ILS-10-2020-0233

Moon J, Jeong M, Oh S, Laine T, Seo J. Data collection framework for context-aware virtual reality application development in Unity: Case of avatar embodiment. Sensors. 2022;22(12).https://10.3390/s22124623

Pratviel Y, Bouni A, Veronique DA, Larrue F, Arsac LM. Avatar embodiment in VR: Are there individual susceptibilities to visuo-tactile or cardio-visual stimulations? Frontiers in Virtual Reality. 2022;3.https://10.3389/frvir.2022.954808

Salagean A, Crellin E, Parsons M, Cosker D, Fraser DS. Meeting your virtual twin: Effects of photorealism and personalization on embodiment, self-identification and perception of self-avatars in virtual reality. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ‘23); 2023. p. 1-16. https://10.1145/3544548.3581182

Škola F, Tinková S, Liarokapis F. Progressive training for motor imagery brain-computer interfaces using gamification and virtual reality embodiment. Frontiers in Human Neuroscience. 2019;13. https://10.3389/fnhum.2019.00329

Ao D, Wu W, Guo X. Immersion and intersectionality - Virtual reality in cross cultural art exhibition courses. In: Rau PLP, editor. Cross-Cultural Design. HCII 2023. Lecture Notes in Computer Science, vol. 14024. Springer; 2023. https://10.1007/978-3-031-35946-0_16

Bowman DA, McMahan RP. Virtual reality: How much immersion is enough? Computer. 2007;40(7):36-43. https://10.1109/MC.2007.257

Moffitt RL. A psychosocial investigation of exercise preferences in real and virtual environments. Psychology of Sport and Exercise. 2023; 70:102530. https://10.1016/j.psychsport.2023.102530

Gammage KL, Lamarche L, Drouin B. Self-presentational efficacy: Does it moderate the relationship between social physique anxiety and physical activity in university students? International Journal of Sport and Exercise Psychology. 2014;12(4):357-367. https://10.1080/1612197X.2014.932824

Dirin A, Laine T. The influence of virtual character design on emotional engagement in immersive virtual reality: The case of feelings of being. Electronics. 2023;12(10). https://10.3390/electronics12102321

Gani A, Pickering O, Ellis C, Sabri O, Pucher P. Impact of haptic feedback on surgical training outcomes: A randomised controlled trial of haptic versus non-haptic immersive virtual reality training. Annals of Medicine and Surgery. 2022;83. https://10.1016/j.amsu.2022.104734

Kanakamedala CA, Dankert FJ, Parola R, Egol AK, Aggarwal KV, Lajam MC. Haptic feedback during virtual reality training significantly improves first-year orthopedic resident performance at tibia drilling: A randomized trial. Current Orthopaedic Practice.2023;34(5):251-256. https://10.1097/BCO.0000000000001223