Computer Aided Surgery 4:281?285 (1999) Brief Technical Report Real-Time Simulation of Tissue Deformation for the Nasal Endoscopy Simulator (NES) Uli Bockholt, Dipl.-Math., Wolfgang Mu?ller, Dipl.-Inf., Gerrit Voss, Dipl.-Inf., Ulrich Ecke, M.D., and Ludger Klimek, M.D. Interactive Graphics Systems Group (GRIS), Darmstadt University of Technology, Darmstadt (U.B.), Department of Visualization and Virtual Reality, Fraunhofer Institute for Computer Graphics (Fraunhofer-IGD), Darmstadt (W.M., G.V.), and Department of Otolaryngology, Head and Neck Surgery, Mainz University Hospital, Mainz (U.E., L.K.), Germany ABSTRACT Endonasal sinus surgery requires a great amount of training before it can be adequately performed. The complicated anatomy involved, the proximity of relevant structures, and the variability of the anatomy due to inborn or iatrogenic variations make several complications possible. Today, cadaver dissections are the ?gold standard? for surgical training. To overcome the drawbacks of traditional training methods, the Fraunhofer Institute for Computer Graphics is currently developing a highly interactive medical simulation system for nasal endoscopy and endonasal sinus surgery, in cooperation with the Mainz University Hospital. For the simulation of a rhinoscopic procedure, not only are the realization of the 3D interaction and the geometric representation of the anatomical structures necessary, but also a real-time simulation of the deformation behavior constrained by the instrument collisions. The challenge is to close the gap between a maximal degree of realism and the required real-time conditions. Comp Aid Surg 4:281?285 (1999). �99 Wiley-Liss, Inc. Key words: surgical simulation, skill training, virtual reality, soft tissue deformation, rhinoscopic education BACKGROUND Endonasal surgery has become standard for the treatment of diseases of the paranasal sinuses and a variety of other pathologies that can be reached via the nasal cavity. In general, the purpose of this surgery is to relieve intractable sinus pain, to remove large expanding mucoceles or pyoceles, and to prevent or control further central, orbital, or external extensions of infection. Since its introduction, functional endoscopic sinus surgery (FESS) has demonstrated success rates of 76% to 98%.3,6 Of the patients who failed to respond to both FESS and initial medical therapy, only a small number require revision endoscopic surgery (RESS).6 The most frequent intraoperative findings, such as ad- Received May 17, 1999; accepted August 23, 1999 Address correspondence to: Uli Bockholt, Interactive Graphics Systems Group (GRIS), Darmstadt University of Technology, Rundeturmstrabe 6, D-64283 Darmstadt, Germany; Telephone: 49-(0)6151-155-283; Fax: 49-(0)6151-155-196. E-mail: email@example.com. This paper is based on a presentation at Medicine Meets Virtual Reality (MMVR) 7, held in San Francisco, California, in January 1999. �99 Wiley-Liss, Inc. 282 Bockholt et al.: Real-Time Simulation of Tissue Deformation hesions, maxillary ostial stenosis, recurrent polyposis, and incomplete removal of diseased air cells, often lead to a loss of normal anatomic landmarks. Therefore, the surgeon must depend on his experience and knowledge of more general anatomic relationships. The ?gold standard? today remains cadaver dissection. Unfortunately, only a limited number of cadavers are available for each trainee, and some of these trainees may go on to perform surgery in patients before having gained sufficient surgical skills. Moreover, it is most likely that the trainee will not see rare anatomic variations or perform revision surgery in a cadaver: A normal variant without disease will be found in most specimens, the tissue is often changed by formalin preservation, and no bleeding occurs. To overcome the limitations of these traditional training methods, VR training simulators are being developed for several endoscopic procedures.4,5 Simulation of surgery in a virtual environment for educational purposes has several advantages over cadaver dissection: VR simulators allow for unlimited numbers of procedures with a single system, every anatomic variation can be simulated, and conditions like massive polyps or a postoperative situation with missing landmarks can be included in the virtual environment. METHODS The Nasal Endoscopy Simulator (NES) prototype consists of a graphics workstation (SGI 02), a tracking system, surgical instruments, and a plastic model of the head (Fig. 1). The visual feedback and the control of the training session is realized by a graphical user interface, which includes the possibility to record and replay a training session (Fig. 2). During navigation in the virtual endonasal sinus system with the endoscope, the trainee is able to test various endoscope optics. Collisions between instruments and anatomical structures are detected, and the trainee is able to deform soft tissues with the instruments. In the development of the Nasal Endoscopy Simulator (NES) several goals have to be achieved: ? Patient-specific CT slices have to be reconstructed to form a 3D representation of the anatomical structures. For performance reasons, triangulated surface models are used for the geometric representation (Fig. 3). The trainee can choose several cases with different Fig. 1. VR interface of NES. pathologies. To give the geometric models a realistic appearance, rhinoscopic live images are mapped onto the surfaces. ? The 3D interaction has to be realized, i.e., the different interaction capacities of the typical scissors, gators (biopsy forceps), and absorbers. We use a electromagnetic tracking system to register position and orientation of the surgical instruments. The opening angle of the gator is measured by a small potentiometer (Fig. 4). ? Models have to be developed that describe the relevant physiological behavior of the anatomical structures. For example, the simulation of bleeding and of deformation behavior is important for the endoscopic procedure. In the first step, the simulation of the deformation behavior resulting from the forces exerted by the instruments on the endonasal tissues is realized (Fig. 5). Simulation of Tissue-Specific Deformation Behavior In a rhinoscopic operation, the surgeon is deforming the endonasal tissues with the surgical instruments, e.g., by pulling with scissors or pushing with a probe. To simulate these procedures we have implemented and tested different approaches. The tissue-specific characteristics should be taken into account, the realism of the simulation should be as Bockholt et al.: Real-Time Simulation of Tissue Deformation 283 Fig. 2. Graphical user interface of NES. high as possible, and the real-time condition should not be lost. In the initial approach, the deformation behavior is simulated by some smooth interpolation functions describing the deformed virtual situs. In other approaches, a mass-spring model is used for the simulation. Fig. 3. The virtual situs. 284 Bockholt et al.: Real-Time Simulation of Tissue Deformation Fig. 4. Simulation of surgical instruments. Simulation Using the Smooth Function Method In this approach the deformation behavior is described by ?bump weighting functions?,1 whereby a smaller inner region, surrounding the collision points of the instrument and the anatomical structure, is moved according to the instruments? constraint. The outer region of the surface at a distance from the collision points remains undeformed. A smooth interpolation function (?bump weighting function?) is then used to interpolate between these two regions. According to this interpolation function, the nodes in the area between the inner and outer regions are moved, so that the smoothness of the surface is obtained. Tissue-specific deformation characteristics can be considered in this approach by means of the inner and outer radii of the deformed and undeformed regions, by the shape of these regions, and by the interpolation function. The advantage of this method is its ability to provide a real-time simulation. Combined with the collision detection, the simulation of deformation with the bump-weighting function hardly influences the performance of the medical training system. Simulation Using the Mass-Spring System For the simulation of the deformation of anatomical structures, Finite Elements Methods (FEM) are applied. The simulation is very accurate, but the solving of the equations requires much computational power. Consequently, efforts have been made to overcome these drawbacks by using simplified FEM models which can be solved under real-time conditions.2 Mass-spring systems represent such simplified FEM models connecting two mass points by a spring. In this way, a physiological model of the virtual situs is generated via mass points and springs describing the elastodynamic behavior of the anatomical structures. The elasticity of the spring is described by spring constants, and the masses of the points are controlled by the mass Fig. 5. Simulation of 3-D interaction (left: pulling, right: pushing). Bockholt et al.: Real-Time Simulation of Tissue Deformation values. In the first step, the mass points are attached to the surface nodes, and the springs to the surface edges. In addition, some springs are positioned through the deformable volume, connected to mass points on the opposite side of the surface. The system is initialized in such a way that the energy minimum of the mass-spring model represents the undeformed shape of the physiological model. When surgical instruments are manipulating the surface of the anatomical structure, the colliding faces are deformed in response to the external stimuli with a physically-based behavior. In this case the equilibrium of the mass-spring model is disturbed, and the motion equation has to be solved for each mass point. The solving of the differential equation is realized by the Adams Bashforth method. This high-order approach requires the computationally expensive functional iteration only once. In the second step, the Delaunay triangulation is used to establish a full volume mass-spring model represented as consisting of many more mass points. The uses of such a complex model are manifold, but it needs much more computational power. Moreover, the elastodynamic characteristics of the tissue types can be described by a variety of parameters using a mass-spring model, e.g., spring constants, mass values, and the relevant damping constant of the springs. DISCUSSION The nasal endoscopy training simulator (NES) represents an advanced training system incorporating Virtual Reality and multimedia for training and for quality control in endonasal sinus surgery. The trainees are able to practice various surgical techniques without having to advance their learning curve on humans. The simulation of deformation is a step towards interactive realism in computerassisted training. Current work focuses on the integration of a haptic device, in order to feel the give and resistance of the anatomical structures. In that context, an adequate and more sophisticated description of the tissue-specific elastodynamic characteristics is necessary. 285 In addition, the simulation of virtual cutting is under development. These medical simulators are on the way to founding an educational base which will perhaps be as important to surgery as flight simulators are to aviation. ACKNOWLEDGMENTS We thank Prof. Dr. h.c. Dr.-Ing. Jose? L. Encarnac?a?o and Prof. Dr. med. Wolf Mann for providing the environment in which this work was possible. We also thank all our colleagues and students at our laboratory, especially Kristina Wittig and Harald Hechler; without their work we would not have been able to achieve the results presented herein. Part of this work was funded by the German Research Society (Deutsche Forschungsgemeinschaft) DFG. REFERENCES 1. Bryson S. Paradigms for the shaping of surfaces in a virtual environment. In: SIGGRAPH ?92, 19th International Conference on Computer Graphics ?Interaction Techniques?, Course Notes 9, 1992. 2. Deussen O, Kobbelt L, Tu?cke P. Using simulated annealing to obtain good nodal approximations of deformable bodies. Proceedings of Sixth Eurographics Workshop on Simulation and Animation, September 1995. 3. King JM, Caladarelli DD, Pigato JB. A review of revision functional sinus surgery. Laryngoscope 1994; 104:404 ? 408 4. Ku?hnapfel U, Krumm H-G, Kuhn C, Hu?bner M, Neisius B. Endosurgery simulations with KISMET. Virtual Reality World?95, Stuttgart, 1995. 5. Mu?ller W, Bockholt U. The Virtual Reality Arthroscopy Training Simulator. In: Westwood JD, Hoffman HM, Stredney D, Weghorst SJ (eds): Proceedings of Medicine Meets Virtual Reality 6, San Diego, CA, January 1998. Amsterdam: IOS Press; 1998. p 13?19. 6. Schaitkin B, May M, Shapiro A, Fucci M, Mester SJ. Endoscopic sinus surgery: 4-year follow-up on the first 100 patients. Laryngoscope 1993;103:1117? 1120.