Computer Aided Surgery 4:65?76 (1999) Biomedical Paper Fluoroscopy as an Imaging Means for ComputerAssisted Surgical Navigation R. Hofstetter, Dipl.-Ing., M. Slomczykowski, M.D., Ph.D., M. Sati, Ph.D., and L.-P. Nolte, Ph.D.-Ing. Maurice E. Mu?ller Institute for Biomechanics, University of Bern, Bern, Switzerland ABSTRACT Objective: Intraoperative fluoroscopy is a valuable tool for visualizing underlying bone and surgical tool positions in orthopedic procedures. Disadvantages of this technology include the need for continued radiation exposure for visual control, and cumbersome means of alignment. The purpose of this article was to highlight a new concept for a computer-assisted freehand navigation system that uses single intraoperatively acquired fluoroscopic images as a basis for real-time navigation of surgical tools. Materials and Methods: Optoelectronic markers are placed on surgical tools, a patient reference, and the fluoroscope to track their position in space. Projection properties of the fluoroscope are acquired through an initial precalibration procedure using a tracked radiopaque phantom grid. Corrections are applied to compensate for both the fluoroscope?s image intensifier distortions and the mechanical bending of the C-arm frame. This enables real-time simulation of surgical tool positions simultaneously in several single-shot fluoroscopic images. In addition, through optoelectronically tracked digitization of a target viewpoint, the fluoroscope can be numerically aligned at precise angles relative to the patient without any X-ray exposure. Results: This article shows the feasibility of this technology through its use in cadaver trials to perform the difficult task of distal locking of femoral nails. Comp Aid Surg 4:65?76 (1999). �99 Wiley-Liss, Inc. Key words: C-arm, fluoroscopic image registration, computer-integrated surgery, distal locking, femoral nail INTRODUCTION Mobile fluoroscopic devices (C-arms) are an integral part of the standard equipment used in orthopedic surgery to provide real-time feedback of bone and surgical tool positions. A constant or pulsed mode allows control of surgical actions, and single buffered images can be used for diagnosis and verification. Although C-arms provide very valuable in situ image information, their use results in exposure of the patient and operating room staff to radiation. This is especially true when constant and higher-frequency pulsed modes are used for longer periods of time while the surgeon is operating close to the field of view. Since fluoroscopy is a twodimensional (2-D) imaging technique, it cannot provide depth information within a single image. This makes the control of complex three-dimensional (3-D) manipulations difficult and requires repeated use of the fluoroscope at different viewing angles. Currently available computer-assisted ortho- Received October 23, 1998; accepted May 3, 1999. Address correspondence/reprint requests to: R. Hofstetter, Maurice E. Mu?ller Institute for Biomechanics, University of Bern, Murtenstrasse 35, CH-3010 Bern, Switzerland. E-mail: email@example.com. �99 Wiley-Liss, Inc. 66 Hofstetter et al.: Fluoroscopy for Surgical Navigation pedic surgery systems are generally based on 3-D image data sets that are acquired preoperatively in a computed tomography (CT) scanner. Multiple views of the imaged anatomy are then generated and brought to the computer screen where the positions of surgical tools are represented.1,2 These systems provide the missing link between the anatomy and medical images by visualizing the positions of surgical tools relative to the image data, and they have proven to be very useful for certain orthopedic interventions such as pedicle screw insertion. However, for other surgeries, e.g., in the field of osteosynthesis, it is often not possible to justify acquisition of a CT data set because of both cost and time constraints. This is especially true for distal locking of the unreamed femoral nail, one of the most difficult steps in the process of femoral fracture reduction. During this closed procedure, the C-arm is presently the only aid to guiding a surgical drill through the locking holes of the intramedullarly placed nail. In the first step of the conventional approach, an accurate C-arm alignment has to be achieved to obtain a view through the locking holes, reproducing them as perfect circles. This trial-and-error method requires patience and sound experience in C-arm handling. Next, a special radiolucent drill is guided under constant C-arm control through the bone and the locking holes. Note that here the surgeon is performing a 3-D surgical action using only one 2-D view, which often leads to errors. Furthermore, these two steps are responsible for a high percentage of the total X-ray exposure during this procedure. This has led us to integrate the C-arm into a computer-assisted surgery system and to provide surgical navigation based on intraoperatively acquired single-shot images. Two different features required for the described locking procedure have been included: navigation of surgical tools in Carm images, and navigation of the C-arm to obtain optimal views of the anatomy. The concept required the development of an accurate method for automatic registration of fluoroscopic images which did not interfere with routine operation procedures. This article describes the general methods of this technology and demonstrates its feasibility for use in locking femoral nails. passive robotic manipulator was used to measure the positions of surgical tools and a phantom for C-arm image registration. For a project involving total hip replacement,4 the use of registered C-arm images has been proposed for coregistering CTimage data or 3-D models to a surgical robot. Hamadeh et al.5,6 and Weese et al.7 used several images of a calibrated C-arm for coregistration of spinal CT data. Brack et al.8 used a calibrated C-arm to guide saw cuts for knee arthroplasty surgery; Brandt et al.9 proposed the use of intraoperatively acquired and registered C-arm images to guide active or semiactive surgical robots in positioning orthopedic implants. Viant et al.10 recently published a method based on biplane-registered C-arm images for determining the trajectories of distal locking screws through a femoral nail. In all of these papers, C-arm calibration is performed separately for each intraoperative image by placing a calibration phantom in the field of view. We have chosen an approach involving C-arm precalibration that avoids the use of calibration phantoms in every image and is thus advantageous for routine clinical use. A first prototype using our concept was briefly described in 1997.11 Because the C-arm?s X-ray projection can be modeled as an optical camera system (extrinsic calibration), our work has been inspired by the calibration of video cameras, especially in industrial and robotic applications where much work has been done.12 Martins introduced an extrinsic biplane calibration method using coplanar-placed calibration markers and studied its accuracy.13 A detailed mathematical approach for calibrating a video camera using a pinhole camera model was given by Gembran et al.,14 and a more complicated method using B-spline instead of linear approximation was proposed by Champleboux and colleagues.15 Accurate calibration of the C-arm requires consideration of distortions caused by the image intensifier process that can be separately modeled (intrinsic calibration). In earlier published reports,16 ?18 pincushion distortions were handled through a few parameters of a physical model and methods for correction were proposed. Mansbach19 developed a method to calibrate an optical system without distinguishing between intrinsic and extrinsic distortions. Previous Work An optoelectronic position sensor (Optotrak 3020; Northern Digital, Waterloo, Ontario, Canada) is used to track the position of surgical tools, a patient reference, and the image intensifier of the C-arm The first attempts to use fluoroscopy in a system for image-guided surgery were made in 1989, when the technique was applied to kidney stone removal.3 A MATERIALS AND METHODS Hofstetter et al.: Fluoroscopy for Surgical Navigation 67 system (P-COS). The transformation matrix TT,P, transforming coordinates from the T-COS to the P-COS, is provided in real time by the position sensor; then, vP is transformed into the C-arm image intensifier coordinate system (A-COS). The transformation matrix TP,A describes the position of the C-arm image intensifier relative to the patient at the time of image acquisition. The position of the C-arm must be acquired directly after its trigger button has been pressed. Note that every C-arm image loaded onto the computer requires a separate TP,A. Finally vA is transformed to vI located in the 2-D, pixel-based C-arm image coordinate system (I-COS). Fig. 1. System setup with components and their associated local coordinate systems (COSs). Top left: C-arm and the image intensifier. Top middle: Position sensor. Top right: Example with a surgical instrument. Bottom left: Patient representing the surgical object. Bottom right: Carm image as displayed on the monitor; this is a 2-D COS. within the region of the operating table. Each component is equipped with infrared (IR) light-emitting diode (LED) markers which define local coordinate systems (COS) (Fig. 1). The position sensor supplies data needed to perform coordinate transformations between these local coordinate systems. This data is passed on to a workstation computer (Ultra 1; Sun Microsystems, Mountain View, CA). An off-the-shelf video framegrabber board (SLICVideo; Osprey Systems, Cary, NC) allows loading of gray-scale images from the video buffer of the C-arm (BV22; Philips, Hamburg, Germany) to the workstation. Our objective was to arrive at a system that works as follows: The surgeon acquires a number of single C-arm views from different orientations that are all displayed on the workstation screen. A computer-generated projection of a surgical tool is then displayed on each image. This is equivalent to its representation under conventional constant fluoroscopic control. An update rate of 10 Hz enables real-time navigation in up to four C-arm images simultaneously. The underlying computations are based on a chain of transformations as given in Equation 1. Let the tip of a drill bit be described by a 3-D vector vT. This vector is constant in its local tool coordinate system (T-COS; note that the subscripts indicate the COS in which the vectors are reported). Considering the position of the surgical object, which can either be the patient or an implant, vT is first transformed to vP in the patient/implant coordinate vT f vP f vA f vI with v P 5 v T z T T,P, v A 5 v P z T P,A, v I 5 v A z T A,I (1) The associated transformation TA,I that models the actual X-ray projection is based on a cone beam projection model as shown in Figure 2. It was used by Mansbach19 to calibrate video cameras for robotic applications. Let the center of the X-ray source be represented by a 3-D vector, fA (focal point). Two vectors, rA and cA, that point along the rising row and column coordinates of the digital image define the orientation of the image plane onto which the C-arm image is projected. The 2-D vector pI, present in the I-COS, represents the piercing point of an image plane normal through fA. To transform a 3-D coordinate vA into a 2-D coordinate vI, a unit vector sA is first calculated Fig. 2. Linear cone beam projection model as used for extrinsic C-arm calibration. The X-ray is emitted at location fA and projects a point vA, representing a tool tip, onto the image plane as vI. pI is defined by the point where rays pass directly normal through the plane. 68 Hofstetter et al.: Fluoroscopy for Surgical Navigation Fig. 3. Setup for the extrinsic C-arm calibration. A plate containing steel spheres is imaged in a proximal and a distal position with respect to the C-arm image intensifier. The 3-D centers of the spheres vAProx, vADist are determined through optoelectronic tracking. (Equation 2) that points in the direction of the appropriate X-ray beam. sA 5 vA 2 fA uv A 2 f Au (2) In a second step, sA is projected onto the image plane through two scalar products (Equation 3). The result is a 2-D coordinate, which is added to the piercing point pI. vIrow 5 sA z rA 1 prow vIcol 5 sA z cA 1 pcol pI 5 S pp D, v 5S vv D row col Irow I (3) Icol The vectors fA, rA, cA, and pI are projection parameters and describe the properties of the Carm. While the vectors rA and cA are assumed to remain constant in the A-COS for the lifetime of the C-arm, we have found that the X-ray source position fA and the related piercing point pI change on most C-arm models owing to mechanic deformations of the C-arm frame. However, all projection parameters are determined in an initial extrinsic C-arm calibration procedure, which is repeated periodically when required. The method used to compensate for the C-arm frame deformations is described below. Extrinsic Calibration The calibration is based on a plate containing radiopaque spherical markers arranged in a rectangular grid. This plate is imaged twice by the C-arm at a proximal and distal position relative to the image intensifier. To obtain the 3-D positions of each sphere, the calibration plate is instrumented with LED markers defining a local COS (C-COS). Three reference indents, iC1, iC2, and iC3, locally define the position of the rectangular grid (Fig. 3). The coordinates of these points are digitized with respect to the C-COS using an optoelectronically tracked pointer. After these incision points have been transformed from the C-COS into the ACOS, the coordinates of all sphere centers are generated. From here on, we will refer to the 3-D sphere centers of the proximal (vAProx) and the distal (vADist) calibration image in the A-COS as the original coordinates. The corresponding image coordinates vIProx and vIDist (I-COS) of the projections of the spheres are obtained through image analysis (Fig. 4). By sorting according to their positions, every image coordinate can be assigned to a corresponding original coordinate. For each calibration image, a 3 3 3 transformation matrix AProx, ADist is set up. This allows transformation of a 2-D image coordinate vI into 3-D original coordinates vAProx and vADist, located in the plane of the calibration plate using the following equations: Hofstetter et al.: Fluoroscopy for Surgical Navigation Fig. 4. C-arm image of the calibration plane in proximal position. The 2-D center coordinates of the sphere markers are automatically detected through image analysis and marked by a white cross. v AProx 5 A Prox z v ADist 5 A Dist ? S D S D v IProxx v IProxy , 1 v IDistx v IDisty 1 (4) AProx is calculated by setting up an overdetermined system of equations using all vAProx and vIProx of a calibration image. A solution with a least-squares error is found using the matrix pseudoinverse: A Prox 5 V AProx z V TIProx z (V IProx z V TIProx)21 (5) where the matrix VAProx contains all column vectors vAProx and the matrix VIProx all column vectors vIProx with the z-components set to 1. The same method is used to calculate ADist. By means of Equation 4, it is then possible to find the corresponding coordinates vAProx and vADist for any vI. The lines formed by joining each vAProx, vADist represent the X-ray beams belonging to each image coordinate vI (Fig. 3). All X-ray beams ideally intersect at the focal point of the system?in this case, the center of the X-ray source fA. This point is found by minimizing its distance to all lines of sight corresponding to the spheres of the proximal calibration image using a least-squares error minimization algorithm.14 The remaining projection parameters, rA, cA 69 and pI, are determined by setting up a system of equations based on Equations 2 and 3, with all pairs of image and original coordinates for both calibration images. A matrix pseudoinverse technique is used to solve this overdetermined system of equations. A detailed description of the mathematics can be found in Gembran et al.14 The errors included during this precalibration step have a direct and systematic influence on the navigation error. Errors are introduced through the two optoelectronic-based coordinate transformations, which are required to measure the 3-D centers of the spheres in the two calibration-plate positions. The tracking system provides coordinate data with a total RMS error of about 0.2 mm per LED marker.20 Roughly estimated for the use of optimized LED marker shields on both the image intensifier and calibration plate, this introduces a mean error of 0.4 mm for navigation within the volume between proximal and distal calibration plates. Additional errors are caused by the image analysis algorithm detecting the centers of the spheres in the calibration images. However, it was found through informal testing that, owing to the high number of markers, this error is negligible. Overall accuracy analysis is described later in System Accuracy Study. C-Arm Frame Deformations Because of the significant weight of the image intensifier and the X-ray source unit, the C-arm frame is subject to variable states of stress according to the C-arm?s orientation. This may cause significant frame deformations when the C-arm is moved. With one of the C-arm models available for our studies, we experimentally measured a maximum movement of fA to be 9.8 mm relative to the A-COS. The C-arm frame must therefore be con- Fig. 5. Position-sensing concept with corresponding coordinate systems to compensate for errors related to C-arm frame deformations. 70 Hofstetter et al.: Fluoroscopy for Surgical Navigation fS in the S-COS, where it is assumed to remain constant. ? With the C-arm in a position (a,b), fS is transformed back from the S-COS into the A-COS using a transformation matrix provided by the position sensor. Fig. 6. tool. Weight-based optoelectronic gravity measurement sidered to act as a nonrigid body with changing projection parameters fA and pI. To avoid related inaccuracies, these deformations are measured using the optoelectronic position sensor. A set of LED markers is attached to the X-ray source chassis of the C-arm (Fig. 5) to define a local coordinate system S-COS. Our strategy is to store deformations preoperatively over all possible C-arm orientations in a lookup table (deformation calibration). During the surgical intervention, when the S-COS markers are not visible for the position sensor, the stored deformation parameters corresponding to a given C-arm position are used to update the extrinsic projection parameters fA and pI. In a preliminary study with two different C-arm models, we found that deformations of the frame caused by changes in its position are within the elastic range of the material, and mechanical parts such as bearings add no significant hysteresis. We therefore assume that the C-arm frame deformation is a repeatable function of its orientation relative to gravity. The spatial orientation of the C-arm relative to gravity can be expressed by two angles, a and b (Fig. 5). An optoelectronic weight-based gravity measurement tool has been designed to provide the system with a vertical gravity vector gA as a reference orientation (Fig. 6). a and b can be calculated from the orientation of this gravity vector relative to the image intensifier. The corresponding focal point position fA(a,b) is measured in the following way: ? At the time of extrinsic calibration, the initial value of fA is transformed from the A-COS to To generate the lookup table, the C-arm is moved in 10� intervals to all orientations required in an operation, and values of a, b, and fA(a,b) are stored. When an image is acquired during an operation, the C-arm orientation angles as and bs are measured. Four entries closest to as and bs are then identified in the lookup table. A bilinear interpolation between the related four values of fA(a,b) is used to compute the focal point fA(as,bs) valid for the currently acquired image. To recalculate the piercing point pI for the new value of fA(as,bs), Equations 2 and 3 are solved for pI with vI 5 (0,0). vA is calculated using Equation 4 for the proximal calibration image. We assume that AProx remains constant for all possible fA(a,b) because the proximal calibration plate is placed directly on the image intensifier input phosphor. Efforts were made to keep errors originating from this method of C-arm deformation compensation at a low level. These errors are mainly caused by the two involved optoelectronic-based coordinate transformations between the A-COS and the S-COS, in conjunction with the large distance of the associated marker shields. We optimized the size and position of these marker shields and experimentally isolated the related errors. We therefore mounted both marker shields at the ends of a nondeforming metal bar at a distance equal to their positions on the C-arm. Ideally, the tracking system should provide a constant position of fA for every orientation of this rigid body in space. With optimized marker shields, we obtained a maximum variation for the position of fA of 0.8 mm.21 Image Intensifier Distortions Video-fluoroscopic images often have distortions that originate from the image intensification process. Two main sources for these distortions have been distinguished: ? Stationary pin-cushion distortions arise from both the spherical shape of the image intensifier input phosphor, onto which the X-rays are projected, and the optical characteristics of the video camera. Hofstetter et al.: Fluoroscopy for Surgical Navigation Fig. 7. Triangle-based displacement interpolation for distortion correction. Correction vector from pixel inside of top left triangle obtained from linear interpolation of vertex corrections. ? Distortions of more arbitrary shape that depend on the position of the image intensifier relative to the earth?s magnetic field and other external magnetic fields are caused by the electron optics of the photo multiplier.17 Our approach is to undistort the image so that it fits into the linear projection model. The stationary component of the distortion is removed in an intrinsic precalibration procedure using a local linear distortion correction method. The centers of the markers in the proximal extrinsic calibration image represent actual local values of the distortions. Reference values for an undistorted image are calculated by transforming the 3-D coordinates of the spheres forming the grid into the C-arm image (I-COS), using the projection algorithm described above (Eqs. 2 and 3) with the parameters obtained during the extrinsic calibration. This guarantees an exact adjustment of the distortion correction to the extrinsic calibration. For every marker, a displacement vector is calculated that points from a position in a distorted image to a corresponding position in an undistorted image. A bilinear local interpolation is performed for every pixel of the image to identify its displacement. The grid of distorted markers is therefore divided into triangles as shown in Figure 7. The displacement of every pixel within a triangle is 71 interpolated using displacement values of the markers at the three edges and is then rounded to a nearest integer neighbor. For reasons relating to computing speed, the displacement of all C-arm image pixels is precalculated and stored in a table, and this is applied to every image obtained during an operation. Remaining gaps are finally filled using linear gray-value interpolation. When applying this method, errors are caused by the fact that nonlinear distortions are compensated for using linear interpolation. To ascertain this error experimentally, we imaged a phantom with 30 steel spheres (2 mm in diameter) arranged in a straight line, and applied the described distortion correction algorithm. Image analysis software was then used to determine the centers of the spheres in the corrected image. An ideal straight line was obtained from these data using a linear regression algorithm. The mean deviation of the preserved coordinates from this ideal line was 1.2 pixels, with a standard deviation of 1.0 pixel for a total of five trials. Further errors originate from distortions caused by external magnetic fields, which cannot be compensated for by precalibration. They appear once the C-arm is moved to positions differing from that at the time of calibration, however, the C-arm under investigation here was not very sensitive to magnetic field distortions. The effect of this error on overall system accuracy is reported in System Accuracy Study. C-Arm Alignment A major difficulty in handling a C-arm during an operation is ensuring its accurate alignment relative to the patient?s anatomy to obtain the desired view. In current clinical practice, this is accomplished by repeatedly acquiring verification images or by using the constant imaging mode. To reduce the resultant radiation exposure, we propose a computerbased C-arm alignment method. The basic concept is to define the desired C-arm orientation with respect to a patient reference using a pointing device. Defined points are represented by a graphical object that is rendered through the cone beam projection model onto a virtual C-arm image. This preview mode allows C-arm alignment without radiation exposure. For example, two C-arm views must be well aligned when preparing the holes to lock an unreamed femoral nail. One view must be oriented exactly through the locking holes so that they appear as ideal circles, while the second view has to be aligned perpendicular to this hole. In our system, 72 Hofstetter et al.: Fluoroscopy for Surgical Navigation in which the surgeon can control the position of the active surgical tool. A zoom on two of these images can be seen in Figure 9, showing the pseudo 3-D navigation obtained through simultaneous multiplanar views. Note that the drill is represented as a correspondingly thick line, and that an optional ?trajectory line? (dotted line) helps surgeons predict where the drill will pass with the current orientation. The system can be easily configured to visualize any defined landmark (Fig. 8) and can be used to navigate within any rigid bone. It is therefore a generalized tool, and is rapidly adaptable to new applications. System Accuracy Study Fig. 8. Graphical user interface allowing precise C-arm alignment. Holes 1 and 2 are represented by small circles about the nail axis. The nearly horizontal line represents the nail?s long axis. The larger circle represents the C-arm?s field of view. In addition, analog and numerical displays (bottom) indicate the angular position of the C-arm relative to a hole axis for fine-tuning. the patient/implant reference is preoperatively fixed to the nail (nail reference). Prior to nail insertion, the positions and orientations of the locking holes are defined with respect to the nail reference using a special optoelectronic pointing tool. When the nail has been implanted and the C-arm is ready to be aligned, these landmarks are visualized in realtime before obtaining an image. Each locking hole is represented by a pair of small circles, a long straight line specifies the position of the nail axis (Fig. 8), and the large circle predicts the field of view of the C-arm image. A perfect C-arm alignment is achieved when two circles belonging to a hole are matched and located in the center of the field of view. Fine-tuning of the C-arm rotations can be performed through feedback from the two sliders at the bottom of the interface. Once an image aligned directly through the hole is obtained (Fig. 9, right), the system can be switched to guide the C-arm view perpendicular to the hole axis (Fig. 9, left). To verify the total system accuracy, i.e., the accuracy with which a surgical tool is represented on acquired C-arm images, a dedicated optoelectronically tracked phantom was constructed. This accuracy phantom contains eight spherical steel markers (2 mm in diameter), aligned in two planes (Fig. 10). The coordinates of the sphere centers relative to the optoelectronic markers were measured with a maximum error of 0.2 mm. Imaging this phantom allows verification of the system accuracy by comparing the actual sphere representations to those calculated by the system. For visual estimation of the errors, small crosses are superimposed on the image in the calculated sphere center position: If the cross is located outside the associated sphere, the error is .1 mm. An exact error quantification, as required for an accuracy study, is performed by metric measurements on image printouts. The C-arm used for the accuracy study (Comet Telam C125, Liebefeld, Switzerland) was calibrated using a calibration plate with 137 markers and a grid width of 10 mm. We then acquired a total of 40 images with the C-arm in various positions and at various distances from the described User Interface A simple graphical user interface was implemented to allow easy use of the system in the operating theater. The image acquisition mode enables the user to load and display up to nine C-arm images. Four of these are displayed in the navigation mode Fig. 9. Real-time image interactive guidance of surgical drill used to distally lock femoral intramedullary nails. The left view shows the countersinks and the right view is aligned straight down the axis of the hole. Hofstetter et al.: Fluoroscopy for Surgical Navigation Fig. 10. Optoelectronically tracked accuracy verification phantom. The representations of contained steel spheres in an X-ray image are compared to those determined by the system. phantom covering the range of its intraoperative use. This led to a total of 232 visible spheres providing an equivalent number of error values. We evaluated all of these visible spheres, including those on the border outside the range of the distortion correction. The resulting mean error was 0.55 mm, with a standard deviation of 0.47 mm and a maximum value of 2.34 mm. The error distribution is given in Figure 11. Laboratory Pilot Study Distal locking of femoral nails was chosen to demonstrate the feasibility of this technology. A wide 73 variety of tools and surgical techniques, such as locked mechanical devices, guidance systems, fluoroscopic control, and radiolucent drills, have previously been proposed for the accurate and safe placement of the locking screws. The key problem lies in the significant deformation of the distal end of the nail upon insertion into the femur.22 As a result, the extensive near-real-time use of the Carm is commonly accepted. However, besides causing significant radiation exposure to the patient and staff, the surgical outcome is still not optimal, since a direct link between the intraoperative images and the surgical action cannot be provided with current techniques. Using our proposed system, we suggest the following modified surgical procedure: Before insertion of the nail, a dynamic reference base with LEDs is attached to the proximal end of the nail. The position of every distal hole is defined with a pointing tool, and computer-aided C-arm alignment ensures images are aligned with the axes of the locking holes (Fig. 9, right). The computer guides the C-arm through a 90� rotation and a second image is taken on which the counter-sinks of the holes can be easily identified (Fig. 9, left). Realtime image-interactive navigation of a surgical tool is now available to the surgeon: The position of the drill can be controlled in two images simultaneously, giving a pseudo?3-D guidance, and the locking holes can be safely prepared. No additional Fig. 11. Distribution of the total system error determined in the accuracy study with a total of 232 samples in 40 C-arm shots. 74 Hofstetter et al.: Fluoroscopy for Surgical Navigation fluoroscopy updates are necessary, resulting in significantly reduced radiation exposure. Note that precise computer-guided alignment of the C-arm with respect to the locking holes is done under the assumption that deformations applied to the nail during its insertion do not cause significant rotation to the axes of the locking holes or torsion to the nail, but rather lead to translational displacements of the locking holes, which do not significantly affect the C-arm orientation alignment. This assumption has been shown to be true for the tibial nail in a preliminary study performed in connection with the development of a guidance device.23,24 Furthermore, we have found that small view misalignments due to nail bending are indicated in the C-arm image by a slightly oval appearance of the locking hole, and this can be compensated for during surgery. It is important to note that the image is taken with the nail already bent, so navigation accuracy? unlike C-arm view alignment?is unaffected by this phenomenon. Three test series were carried out: (a) a laboratory study on 25 plastic femurs involving a total of 50 locking holes; (b) an in vitro study using 10 human cadaver femurs with 20 locking holes; and (c) an application on two full cadavers with a total of four locking holes. In all three series, the new system allowed easy and successful insertion of the locking screws. In some cases (20% in series a, 10% in series b, and 0% in series c), the drill touched the nail slightly before entering the hole, with no significant damage or consequences to the locking procedure. The mean X-ray exposure time per pair of prepared holes was 1.78 s for series a, 1.63 s for series b, and 1.65 s for series c. These studies are described in detail in a forthcoming paper (Slomczykowski M, Hofstetter R, Sati M, Krettek C, Nolte L-P. A novel computer assisted fluoroscopy system for intra-operative guidance: feasibility study for distal locking of unreamed femoral nails. Submitted to Journal of Orthopaedic Trauma). DISCUSSION A novel kind of computer-assisted freehand navigation system has been proposed that is based solely on intraoperatively acquired fluoroscopic images. In addition, rapid acquisition of these images is supported by computer-guided alignment of the C-arm. This technology was used to perform computerized distal locking of femoral nails in a laboratory setup on plastic and cadaver bones. The results indicate potential surgical benefits for in vivo use of the system, and the concept of an entirely precalibrated system keeps its intraoperative handling as simple as possible. In the in vitro studies, the prototype was used with caution and a redundant functional check was performed before preparation of each surgical action. Before the concept can be used in clinics on a routine basis, the mechanical and software components must be made more sturdy and reliable to fulfil the practical requirements of an intraoperative tool. Hardware and software must also be adjusted for different C-arm models to prove that the method yields consistent accuracy. A key issue during development of the navigator was the total system accuracy. Because the C-arm was primarily used as an imaging device rather than as a means of measurement, geometric linearity and independence of external influence were not an issue for most C-arm manufacturers. Thus, the C-arm itself has been found to be the main source of inaccuracy in the concept. Deformation of the C-arm frame, as well as image distortions, are reasons for such inaccuracies, and have to be compensated for to achieve reasonable system accuracy. Our compensation methods significantly decrease? but do not eliminate?the errors. With the C-arm model used in this study, remaining errors arising, for example, from variable image intensifier distortions caused by external magnetic fields limited the mean system accuracy to 0.55 mm and led in rare cases to errors of up to 2.34 mm. We are aware that magnetic distortions can be a larger source of error for certain C-arm models; this is the subject of our ongoing work. However, distal locking of the femoral nail (locking hole � 5 mm, hole sink � 6 mm, drill � 4 mm) could successfully be performed with the technique presented here. This is because the specially concave hole sinks and the conical point of the drill mechanically guide the drill through the hole even when the drill is slightly off target. A potentially more accurate solution is to update the intrinsic and extrinsic calibration in every C-arm image using marker phantoms within the operation as proposed in Pietka and Huang,17 and Weese et al.7 However, this has drawbacks in practice, since the optoelectronically tracked, radiolucent phantoms have to be placed within the field of view in the operation. This is cumbersome and, for many orthopedic procedures, impossible: a typical tradeoff between system accuracy and practical usability. Recently, a method for C-arm precalibration was proposed which allows correction for distortions caused by the earth?s magnetic field.25 The C-arm has to undergo a preoperative ?learning? Hofstetter et al.: Fluoroscopy for Surgical Navigation procedure that determines and stores the distortions for all possible C-arm positions relative to the earth?s magnetic field. We recently developed a more practical solution that allows the removal of distortions caused by all kinds of magnetic fields; this work will be presented in a future publication. A second source of error is the position sensor that?although the most accurate available? has limited measurement accuracy for rigid body transformations. Five consecutive coordinate transformations based on the position sensor directly or indirectly affect the accuracy of a tool?s representation in a C-arm image. By increasing the number of LEDs used and prudently positioning them on the associated rigid bodies, we were able to increase the system accuracy. The system presented here differs from that of Viant et al.10 in that the surgeon navigates directly and freehand on multiple computer-aligned C-arm images; the system of Viant et al. uses stereo reconstruction of the locking hole to define a trajectory that is carried out with an electromechanical locking arm. A fluoroscopy-based system can be seen as a complement to CT-based systems. Its advantage of instant availability without preoperative preparations (CT acquisition, segmentation, and registration) allows it to be used for several applications in the field of traumatology. It is also open for use in other fields where acquisition of CTs cannot be justified owing to cost or radiation exposure issues. Since no manual registration procedure is required, fluoroscopy-based navigation may be advantageous in fields where bone structures do not allow recognition of the landmarks needed for CT-image registration. Of particular interest is the potential for minimally invasive procedures. Since the C-arm is precalibrated, a minimally invasive DRB attachment is sufficient for keyhole surgical approaches under multiplanar X-ray control. CT-based methods are more suited in applications where the image quality of the C-arm is not sufficient or where attainable perspective views do not provide enough 3-D information. CONCLUSION A new computer-assisted surgery system has been developed integrating a standard C-arm into a freehand navigation system. To our knowledge, it is the first system allowing surgical navigation based solely on multiplanar static C-arm images. A graphical user interface was developed that allows real-time image-interactive guidance of surgical instruments without further fluoroscopy updates. 75 Based on the results of the technical accuracy study and laboratory pilot study presented in this article, two potential advantages of the new computerbased technique, as compared to current manual techniques, can be identified: ? ? significantly reduced radiation exposure improvement of surgical accuracy and safety. The system holds the promise for a generalized surgical tool that can be adapted to a wide variety of applications in the field of orthopedic surgery and offers the possibility of performing minimally invasive approaches. ACKNOWLEDGMENTS The authors thank Yvan Bourquin and Heinz Waelti for their help in software programming, Professor C. Krettek for performing the cadaver study, and Medivision AG, Oberdorf, Switzerland, for supporting this work by providing electrical and mechanical hardware. REFERENCES 1. 2. 3. 4. 5. 6. 7. Lavalle?e S, Sautot P, Troccaz J, Cinquin P, Merloz P. Computer assisted spine surgery: a technique for accurate transpedicular screw fixation using CT data and a 3-D optical localizer. J Image Guid Surg 1995;1:65? 72. Nolte LP, Visarius H, Arm E, Langlotz F, Schwarzenbach O, Zamorano L. Computer-aided fixation of spinal implants. J Image Guid Surg 1995;2:88 ?93. Potaminos P, Davies BL, Hibberd RD. A robotic system for minimal access surgery. Proc Inst Mech Eng 1994;208:119 ?126. Joskowicz L, Taylor RH, Williamson B, Kane R, Kalvin A, Gue?ziec A, Taubin G, Funda J, Gomory S, Brown L, McCarthy J, Turner R. Computer integrated revision total hip replacement surgery: preliminary report. In: Proceedings of the 2nd MRCAS Symposium, 1995. p 193?202. Hamadeh A, Sautot P, Lavalle?e S, Cinquin P. Towards automatic registration between CT and X-ray images: cooperation between 3D/2D registration and 2D edge detection. In: Proceedings of 2nd MRCAS Symposium, 1995. p 39 ? 46. Hamadeh A, Lavalle?e S, Cinquin P. Automated 3-dimensional computed tomographic and fluoroscopic image registration. Comp Aid Surg 1998;3:11?19. Weese J, Buzug TM, Lorenz C, Fassnacht C. An approach to 2D/3D registration of a vertebra in 2D X-ray fluoroscopies with 3D CT Images. In: Troccaz J, Grimson E, Mo?sges R, editors. Proceedings of the 1st Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery (CVRMed-MR- 76 Hofstetter et al.: Fluoroscopy for Surgical Navigation CAS?97), Grenoble, France, March 1997. Berlin: Springer-Verlag, 1997. p 119 ?128. 8. Brack C, Go?tte H, Gosse? F, Moctezuma J, Roth M, Schweikard A. Towards accurate X-ray-camera calibration in computer-assisted robotic surgery. In: Lemke HU, Vannier MW, Inamura K, Farman AG , editors. Proceedings of the International Symposium on Computer and Communication Systems for Image Guided Diagnosis and Therapy (CAR?96), Paris, June 1996. Amsterdam: Elsevier, 1996. p 721?728. 9. Brandt G, Radermacher K, Lavalle?e S, Staudte H-W, Rau G. A medical robot system for orthopedic interventions. In: Lemke HU, Vannier MW, Inamura K, editors. Proceedings of the 11th International Symposium and Exhibition on Computer Assisted Radiology and Surgery (CAR?97), Berlin, June 1997. Amsterdam: Elsevier, 1997. p 950 ?955. 10. Viant WJ, Phillips R, Griffiths JG, Ozanian TO, Mohsen AMMA, Cain TJ, Karpinski MRK, Sherman KP. A computer assisted orthopaedic surgical system for distal locking of intramedullary nails. Proc Inst Mech Eng 1997;211:293?299. 11. Hofstetter R, Slomczykowski M, Bourquin I, Nolte L-P. Fluoroscopy based surgical navigation: concept and clinical applications. In: Lemke HU, Vannier MW, Inamura K, editors. Proceedings of the 11th International Symposium and Exhibition on Computer Assisted Radiology and Surgery (CAR?97), Berlin, June 1997. Amsterdam: Elsevier, 1997. p 956 ?960. 12. Tsai RY. An efficient and accurate camera calibration technique for 3D machine vision. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1986. p 273?280. 13. Martins HA, Birk JR, Kelley RB. Camera models based on data from two calibration planes. Comput Graphics Image Process 1981;17:173?180. 14. Gembran KD, Thorpe CE, Kanade T. Geometric camera calibration using systems of linear equations. In: Proceedings of IEEE Conference on Robotics and Automation, 1988. p 562?567. 15. Champleboux G, Lavalle?e S, Sautot P, Cinquin P. Accurate calibration of cameras and range imaging sensors: the NPBS method. In: Proceedings of IEEE 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. Conference on Robotics and Automation, 1992. p 1552?1557. Boone JM, Seibert JA, Blood W. Analysis and correction of imperfections in the image intensifierTV-digitizer imaging chain. Med Phys 1991;18: 236 ?242. Pietka E, Huang HK. Correction of aberration in image intensifier systems. Comput Med Imaging Graphics 1992;16:253?258. Rudin S, Bednarek DR, Wong R. Accurate characterization of image intensifier distortion. Med Phys 1991; 18:1145?1151. Mansbach P. Calibration of a camera and light source by fitting to a physical model. Comput Vis Graphics Image Process 1986;35:200 ?219. Northern Digital, Inc. Optotrak 3020, the 3D motion measurement system. System Specifications, April 1994. Schauer D. Entwicklung von Instrumenten und Apparaten fu?r die computerassistierte orthopa?dische Chirurgie. Diploma thesis, Institute for Electronics, TU Berlin, 1997. Knudsen C, Grobler G, Close R. Inserting the distal screws in a locked femoral nail. J Bone Joint Surg 1991;73-B:600 ? 601. Krettek C, Mann� J, Ko?nemann B, Miclau T, Schandelmaier P, Tscherne H. The deformation of small diameter solid tibial nails with unreamed intramedullary insertion. J Biomech 1997;30:391-?394. Krettek C, Ko?nemann B, Miclau T, Ko?lbli R, Machreich T, Kromm A, Tscherne H. A new mechanical aiming device for the placement of distal interlocking screws in femoral nails. Arch Orthop Trauma Surg 1998;117:147?152. Brack C, Burgkart R, Czopf A, Go?tte H, Roth M, Radig B, Schweikard A. Accurate X-ray-based navigation in computer-assisted orthopaedic surgery. In: Lemke HU, Vannier MW, Inamura K, Farman AG, editors. Proceedings of the 12th International Symposium and Exhibition on Computer Assisted Radiology and Surgery (CAR?98), Tokyo, June 1998. Amsterdam: Elsevier, 1998. p 716 ?722.