A Natural Interaction Paradigm to Facilitate Cardiac Anatomy Education using Augmented Reality and a Surgical Metaphor

Abstract

This paper presents a design approach to creating a learning experience for cardiac anatomy by providing an interactive visualisation environment that uses Head Mounted Display (HMD)-based Augmented Reality (AR). Computed tomography imaging techniques were used to obtain accurate model geometry that was optimised in a 3D modelling software package, followed by photo-realistic texture mapping using 3D painting software. This method simplifies the process of modelling complex, organic geometry. Animation, rendering techniques, and AR capability were added using the Unity game engine. The system's design and development maximises immersion, supports natural gesture interaction within a real-world learning setting, and represents complex learning content. Hand input was used with a surgical-dissection metaphor to show cross-section rendering in AR in an intuitive manner. Lessons learned from the modelling process are discussed as well as directions for future research.