Problem Description

The secret to relevant medical imagery is below the surface. As light shines onto tissue, some of it enters the skin and bounces around inside. This causes the light to scatter in all directions, and escape at points that are a distance away from where they entered, creating a soft, warm glow that appears to come from within the skin. This effect is called subsurface scattering, and simulating this complex phenomenon is crucial when recreating organic materials in film, video games, and medical training. Without accurately simulating subsurface scattering, medical imagery risks not only being inaccurate, but also potentially misleading.

Project Definition

Immersive patient simulators (IPSs) combine the simulation of virtual patients with a three-dimensional (3D) environment and, thus, allow an illusionary immersion into a synthetic world, similar to computer games. Playful learning in a 3D environment is motivating and allows repetitive training and internalization of medical workflows (ie, procedural knowledge) without compromising real patients.

The University Hospital Cologne (Uniklinik) develops and uses surgical training applications in immersive VR environments. The goal of this Guided Project ist to compare physically-based rendering approaches to traditional rendering of organs, especially the liver, in VR applications. The project entails:

• Developing a prototypical VR application using physically based materials, e.g. the High Definition Render Pipeline in Unity.
• Work with medical experts from the University Hospital to enhance existing 3D models to exhibit realistic lighting effects, such as subsurface scattering.
• Perform a user study comparing the effects on participants using traditional rendering and physically based rendering.

Learning Outcome

Students learn to develop applications in VR featuring modern rendering pipelines that enable realistic, physically-based lighting for e.g. subsurface scattering, translucency, iridescence, and anisotropy. Students learn to evaluate different rendering techniques, from traditional rasterization, ray marching and ray tracing (SSAO, SSGI, SSR, Contact or ray traced shadows), to full-frame path tracing, given hardware limitations of mobile VR environments.

Participation Requirements

Students benefit from experience with:

• Real-time 3D rendering and graphics programming
• 3D modeling (e.g. Blender)
• 3D game engines such as Unity / Unreal, and programming concepts (C# / C++)

External Partner

Priv.-Doz. Dr. med. Rabi Raj Datta Facharzt für Viszeralchirurgie | Spezielle Viszeralchirurgie Klinik und Poliklinik für Allgemein-,Viszeral-,Thorax- und Transplantationschirurgie Universitätsklinikum Köln