Virtual Reality and AI for Medical Education

Virtual Reality (VR) is a rapidly emerging technology in medical education that offers unprecedented opportunities for immersive and interactive learning. By simulating realistic clinical environments, VR enables students to engage in examination scenarios at an early stage of their studies, fostering the development of essential diagnostic and procedural skills.

This approach provides students with safe learning environment in which they can refine their competencies without putting real patients at risk, thereby increasing both confidence and knowledge.

By integrating advanced artificial intelligence (AI) models, including large language models (LLMs) and image synthesis methods like diffusion models, we are able to further enhance the educational potential of VR by enabling the generation of customizable clinical scenarios and dynamic, lifelike patient interactions. This synergy between VR and AI not only personalizes the learning experience but also ensures a more comprehensive and adaptive approach to medical education.
 
In collaboration with our local medical experts from the IfAS, we've already implemented several projects in this area and integrated them into the medical curriculum at various German universities. See below for some impressions of these projects.

Virtual Brain Death Examination

Related Publications:

  • Pielage, Leon, Schmidle, Paul, Marschall, Bernhard, and Risse, Benjamin. . “Interactive High-Quality Skin Lesion Generation using Diffusion Models for VR-based Dermatological Education.” in IUI '25: Proceedings of the 30th International Conference on Intelligent User Interfaces, edited by Association for Computing Machinery. New York, NY: ACM Press. doi10.1145/3708359.3712101.
  • , , , et al. . “Teach the unteachable with a virtual reality (VR) brain death scenario - 800 students and 3 years of experience.Perspectives on Medical Education, 14 (1): 4554. doi: 10.5334/pme.1427 .
  • , , , et al. . “VR-based Competence Training at Scale: Teaching Clinical Skills in the Context of Virtual Brain Death Examination.Proceedings of the ACM on Human-Computer Interaction, 8 261. doi: 10.1145/3664635.
  • Junga, A., Schmidle, P., Pielage, L., Schulze, H., Haetscher, O., Staender, S., Marschall, B., & Braun, S. A. (). New horizons in dermatological education: Skin cancer screening with virtual realityJournal of the European Academy of Dermatology and Venereology, 38 (12), 2259–2267. doi10.1111/jdv.19960.
  • , , , et al. . “Immersive learning in medical education: analyzing behavioral insights to shape the future of VR-based courses.BMC Medical Education, 2024 (24) 1413. doi: 10.1186/s12909-024-06337-7.
  • , , , et al. . “Integration of VR into Medical Education (Workshop).” in Würtual Reality, edited by 2023 University of Würzburg, Department of Psychology I (Marcusstr. 9-11, 97070 Würzburg). N/A: Selbstverlag / Eigenverlag. doi: 10.25972/OPUS-31720.
  • , , , et al. . “Hirntoddiagnostik in Virtual Reality – was denken Studierende darüber?” contributed to the Jahrestagung der Gesellschaft für Medizinische Ausbildung (GMA) 2023, Osnabrück doi: 10.3205/23GMA227.
  • , , , et al. . “Zusammenhang von Persönlichkeitsvariablen und Leistung in der virtuellen medizinischen Ausbildung.” contributed to the Jahrestagung der Gesellschaft für medizinische Ausbildung (GMA) 2023, Osnabrück doi: 10.3205/23GMA273.
  • , , , and . . “Diffusion Models in Dermatological Education: Flexible High Quality Image Generation for VR-based Clinical Simulations.” contributed to the NeurIPS'23 Workshop: Generative AI for Education (GAIED), New Orleans, Louisiana
  • , , , et al. . “Virtual Reality based teaching – a paradigm shift in education?” contributed to the 73. Jahrestagung Deutsche Gesellschaft für Neurochirurgie, Köln doi: 10.3205/22DGNC538.
  • , , , , , and . . “Towards VR Simulation-Based Training in Brain Death Determination.” in 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), edited by IEEE. New York City: Wiley-IEEE Press.