MICROSCOPY RESEARCH AND TECHNIQUE 33:516-526 (1996) Three-Point Repositioning of Axes: Three-Dimensional Alignment Procedure for Electron Microscope Tomography Using Three Markers R. JONGES, E. DE MOOR, P.N.M. BOON, J. VAN MARLE, A.J.J. DIETRICH, AND C.A. GRIMBERGEN Department of Medical Physics and Informatics (R.J., E.d.M., PN.M.B., C.A.G.), Department of Electron Microscopy (J.v.M.),and Institute of Human Genetics (A.J.J.D.),Academic Medical Center, University of Amsterdam, 1100 DE Amsterdam, The Netherlands KEY WORDS Reconstruction, Alignment, Micrograph ABSTRACT A description is given of a new procedure to align series of tilted graphs, made with an electron microscope, for computer tomographic purposes. The procedure uses the coordinates of three projected markers to calculate parameters needed for the reconstruction. To that end the procedure computes the direction of the tilt-axis, the translation and rotation parameters, the tilt-angle of every micrograph, and the spatial coordinates of the individual markers with their centre of gravity as origin of the coordinate system. A searching technique, based on cross-correlation, is described to locate accurately the micrographs markers. o 1996 Wiley-Liss, Inc. INTRODUCTION Over the last 10 years computer reconstruction in the field of the electron microscopy became a useful tool in acquiring more insight in biological structures. The alignment, i.e., establishing the relation between the micrographs and the specimen, precedes the reconstruction and is therefore of primordial importance. In the literature two main approaches have been described to solve the alignment problem for a series of micrographs: one based on cross-correlation techniques (Farrow and Ottensmeyer, 1993; Frank et al., 1987; Guckenberger, 1982) and one on fiducial markers. Correlation does not need the extra preparation of the specimen with markers, neither do the markers disturb the reconstruction. However, the accuracy of this type of alignment heavily depends upon the density distribution of the object structures and is sensitive to their immediate surroundings, especially a t large tilt-angles. The advantages and disadvantages of both techniques are discussed by Lawrence (1992). Most alignment procedures that use colloidal gold particles are iterative least squares methods (Berriman et al., 1984; Bonnet et al., 1985; Lawrence, 1983; Luther et al., 1988; Olins et al., 1983). In order to obtain a reliable solution, 8 to 15 markers on each micrograph have t o be introduced. Jing and Sachs (1991) describe a linear least squares method allowing partial use of markers when they are not detectable on all the micrographs. In practice it is left to chance whether or not there are enough markers around the object. Olins et al. (1994) therefore digitized the micrographs in two steps: at low resolution, so as to have enough markers for the alignment and at high resolution, to be used for the reconstruction. This study describes a procedure for alignment using solely three markers in the specimen that is tilted around one axis. It iteratively finds the best fit between the calculated projections of a spatial triangle, derived from the non-tilted record, and the triangle projections 0 1996 WILEY-LISS, INC. of the series of micrographs. The procedure converges to a solution which will then have an error homogeneously distributed over all micrographs. Scaling factors are not used because the deformation of the specimen caused by electron radiation, vacuum, and image distortion in the nonlinear regions of the electron microscope (E.M.) are complicated nonlinear phenomena. To allocate coordinates to the markers in the micrographs we describe a cross-correlation searching procedure between a centralized marker model in a circular domain and the object markers. This method is very insensitive to all kinds of disturbing density surroundings, and is therefore better than procedures searching for a maximum peak of grey level. The procedures were applied with satisfying results (see Marle et al., 1995'). MATERIALS AND METHODS Alignment Introduction. In making tilt series with an E.M., one encounters a number of practical problems in connection with the alignment of micrographs. The mechanical tilt-axis seldomly goes through the object of interest in the specimen. Consequently, when tilting the specimen, a manual correction is necessary to compensate for the horizontal shift of the object in order to keep it in the field of view. This introduces some undocumented translations. Moreover, the steps between the E.M. and the digitizer of the micrographs introduce additional translations and rotations. Using colloidal gold particles as markers a t one or at both surfaces of the specimen, the micrographs were Received April 28, 1995;accepted in revised form November 21, 1995. Address reprint requests to R. Jonges, Department of Medical Physics and Informatics,Faculty ofMedicine,University of Amsterdam, PO Box 22700,1100 DE Amsterdam, The Netherlands. THREE-POINT REPOSITIONING OF AXES re-oriented into their correct positions with respect to the tilt-axis. Definitions: axis of the specimen holder (assumed to be parallel to the projection plane). angle between specimen in horixontal tilt-angle position (parallel to the projectionplane) and its tilted position (maximal k60"). zero-record record of the specimen in horizontal position (tilt-angle = 0'). tilt-record projection of the specimen under a tiltangle. rotation-angle angle over which the projection is rotated in the projection plane. tilt-axis Alignment principle. To align a series of micrographs, the following information of each micrograph is required the translation with respect to a common point of the specimen, the direction of the projected tilt-axis and the tilt-angle. The tilt-axis can be freely displaced parallel to itself in this procedure. The micrographs are digitized in the coordinate system of the scanner. In every micrograph the coordinates of three markers are measured where each projection is originating from one of three non-collinear gold particles (spatial triangle) in the specimen. We define a new X,Y,Z coordinate system in which the tilt-axis coincides with the Y-axis and the X-Y plane is parallel to the projection plane. Referring to this coordinate system the unknown parameters of each micrograph (i) are a translation (Ti), a rotation-angle ( p i ) , and a tiltangle (Oil. The centre of gravity of the spatial triangle is chosen as the origin of the coordinate system. By determining the centre of gravity in every projected triangle and translating this common point to the origin, the parameter Tiis eliminated. Now we have t o solve the rotation angles ( p i ) and the tilt-angles (OJ over n records from the projected triangle coordinates. The size and shape of the triangle formed by three projected markers is determined by the position of the originating triangle, the direction of the tilt-axis and the tilt-angle. If we would know the spatial coordinates of the originating triangle (the Y-axis coincides with the tilt-axis) all parameters could be calculated. The tilt-angle is a function of the triangle area and its projected one; the rotation-angle depends on the Y-coordinates of the spatial markers to which the projected markers have to rotate around the origin. The alignment procedure iteratively tries to find the spatial triangle coordinates from the projected ones by using the zero-record as a reference. When tilting the spatial triangle, the projection of its vertices move over lines perpendicular to the tilt-axis. The shape and the area of the projected triangle are uniquely related to the spatial triangle and its tilt-angle. During iteration the projections of each intermediate spatial triangle are calculated. Comparing the shape of the calculated triangle projections with the shape of the projected ones in the micrographs guides the alignment procedure. It suffices here to ex- 517 press the shape of the triangle by the vector lengths of each vertex. Minimalization of the sum of the absolute differences between the calculated vector lengths and those extracted from the micrographs over all the tiltrecords leads the procedure to a unique solution. This approach reduces the number of rotation-angles to 1, namely, po of the zero-record, because the vector lengths are independent of the rotation-angle. Hence, the parameters to be solved for by the alignment procedure are the rotation-angle of the zero-record (pol and the spatial orientation, as expressed by Xfase and Yfase (see Alignment procedure), of the originating triangle. The alignment procedure is split up into two nested iteration loops. The outer loop tries to find the best spatial orientation of the plane through the three spatial markers. Its output is a set of tilt-angles. The inner loop tries t o find the best value for the rotation-angle of the zero-record. Rotation-angle of the zero-record. To determine the rotation-angle of the zero-record we use an iteration procedure for two reasons. First, we expect errors of the measured coordinates of the markers in the micrographs. Without errors two tilt-records are sufficient to calculate the rotation-angle by knowing the tilt-angle and the plane orientation through the spatial markers. Second, it is easy to involve all the projected triangles in the procedure, so the determined rotationangle is an average result. Assume that theoretical precise triangle coordinates of two tilt-records are available, the zero-record (tiltangle = 0") and a second one at tilt-angle 8. Also assume that the spatial triangle is parallel to the projection plane. Let po, qo, and ro be the projections of three points as they appear in the zero-record (Fig. 1).The coordinate system has its origin 0 in the centre of gravity of these points and in this figure the Y-axis and the tilt-axis coincide. Let the dashed triangle (Po,Qo, Ro) be identical to triangle (po, q,,, ro) but in the correct position with respect to the tilt-axis. Tilting over 8 results (infer from the dashed triangle) in triangle (PO, Qe, Re). The triangle (po, qo, ro) is rotated around 0 with a still unkown rotation-angle. Tilting this triangle also over 8 results in triangle ( p e ,q e , re). Iteratively the following steps are performed. 1. Rotate the triangle (po,qo, ro)by a rotation-angle step (normally starting with 0"). 2. Calculate at tilt-angle 8 the vector lengths of go, de, and Po. 3. Comparsth calcuhted vector lengths with the correct ones Po, and Re. 4. Make a new step in po and perform 1 . . . 4 until the comparison results in equal vector lengths. d, This method puts the zero-record into the correct position with respect to the coordinate system. The inner loop of the alignment procedure is based on this method, working on the projections in all the tiltrecords, and produces the rotation-angle po for the zerorecord. This inner loop is called the "basic rotation pro cedure tilt-axis.'' It searches for the minimal sum 01 absolute differences between all the calculated pro- 518 R.JONGElS ET AL. t Y (tilt-axis) Fig. 1. Determination of the rotation-angleof the zero-record. Triangle (po,qo, ro)is the projection of the three markers as taken from the zero-record,having an unknown rotation-anglewith respect to the Y-axis (tilt-axis). Triangle (Po,Qo,R,) is the correctly positioned one with respect to the Y-axis. Subindex (0) indicates the projections at tilt-angle 8. jections and the measured ones. The differences between the idealized example (Fig. 1) using two records and the practical situation are: the spatial orientation of the triangle is not necessarily parallel to the projection plane, and the tilt-angles are not exactly known. These parameters are determined by the outer iteration loop of the alignment procedure. Alignment procedure. To derive the parameters needed for aligning a series of micrographs with three markers we use their yet unknown three-dimensional coordinates. We take the zero-record as a reference. Note that the plane through the three markers may not be parallel to the plane of projection. Let the zerorecord situation be described as in Figure 2a. The points P, z, Qx,y,z, and Rx,y,rrepresent the spatial markers. ?he centre of gravity Ooo,o coincides with the origin of the coordinate system. The tilt-axis coincides with the Y-axis. PkJ,o, and R:J,o are the projections in the X - Y plane of the zero-record. The orientation of the PQR plane is expressed by the angles between the X-axis and the intersecting line with the Z-X plane and between the Y-axis and the intersecting line with the Z-Y plane (Fig. 2b). We refer to these angles as Xfase and Yfase. To obtain the correct marker coordinates of Px,y,, QxJr, and RXy,+, the procedure starts from the coordinates of the zero-record and changes alternately the Xfuse or the Yfuse. The projected triangle PLJ,oQ&,oRi,o is unaffected but the tilt-angles of the tilt-records epen on the Xfuse. The tilt-angle is a cosine function of the calculated projected area in the micrograph and the spatial area at a given Xfuse. The Yfuse has no effect on the tilt-angles because the tiltaxis coincides with the Y-axis and therefore it has no influence on the magnitude of the projected triangle areas. On the other hand the shape of the projected triangles is controlled by the Yfuse. The angles Xfase and Yfuse are part of the outer iteration loop. The inner loop (“basic rotation procedure tilt-axis”) iteratively searches for that rotation angle po by calculating the projections of P, Q, and R a t the intermediate tilt-angles, which minimizes the sum of absolute differences between the measured and calculated projection lengths. Having minimized this sum of differences, another PQR plane orientation is chosen in order to check whether a lower sum of absolute differences can be obtained. At the end of the alignment procedure the orientation of plane PQR with respect to the tilt-axis and the tilt-angles is determined. To prevent the computation of sets of unrealistic tiltangles, the alignment procedure needs an extra precaution. We have chosen to force the sum of the tiltangles to be zero, thus balancing the negative and positive tilt-angle series on either side of the zerorecord. Hence the alignment procedure involves two convergence criteria: 1. The minimal sum of the absolute differences between the calculated marker distances and the measured distances in the micrographs at all the tilt-angles with the centre of gravity as origin. 2. The sum of the tilt-angles is zero. Figure 3 shows the modules of the complete alignment procedure. The modules, enclosed by the dashed line, are part of the “basic rotation axis procedure” and will search for the optimal rotation of the zero-record at every combination of Xfuse and Yfuse considered. Module description Input of the procedure. The coordinates of three projected markers of the zero-record; the origin is their centre of gravity. For every tilt-record the following is calculated: 1. The area of the projected triangle. 2. The projected vector lengths of the three markers. Modules A, B , and C. Given an Xfuse (controlled by module K) module A calculates an area Ss from the area of the zero-record So. ss = so cos (Xfase) Module B calculates all the different tilt-angles (Oil from the cosine relation of the ratio of the projected micrograph areas Si and the area Ss. oi = arccos (2) 519 THREE-POINT REPOSITIONING OF AXES I Fig. 2. a: TrianglePQR and its projectionP'Q'R'at the X-Y plane. the tilt The origin of the coordinate system is the centre of gravity (0); axis coincides with the Y-axis. b: The orientation of the PQR plane is expressed by the variables Xfase and Yfase. The Xfase is the angle between the X-axis and the intersecting line with the Z-Xplane; the Yfuse is the angle between the Y-axis and the intersecting line with the Z-Yplane. Module C subtracts the X fuse, being an offset for all then the angle Ovm between V,,, and the X-Y plane: the tilt-angles, from every angle (Oil resulting in tiltangles with respect to the zero-record. Module D. Without any control over the resulting evm= arctan tilt-angles the alignment procedure has not enough constraint to converge to one single set of tilt-angles. To get tilt-angles that are realistic, there must be a The projected Y coordinate Y',,, is equal to the momenlink to the true tilting stages of the E.M. Balancing the tary Y coordinate of the zero-record and constant for calculated tilt-angle range results in tilt-angles that every tilt. The projected X coordinate is: are comparable to the E.M. tilt-angles. Balancing is x, = v, . cos (e, + oil (6) executed by averaging the tilt-angles and by subtracting the resulting average from every operative tilt-angle. The tilt-angle of 0" belonging to the zero-record with 8, the tilt-angle. Module H . The distances from the centre of gravity also receives this error. The procedure iteratively tries to theprojected marker points P, Q, and R resp. to make this error, an offset angle of the zero-record, 0" again. The balancing principle is explained in tilt-an- OP; ,OQ;,and are calculated for every tilt angle. gle balancing. 2)ltoduleI . The result of this module is the sum (D)of Module E. The X,Y coordinates of the three pro- the absolute differences between the calculated vector jected markers in the zero-record are rotated around lengths (module H)and the measured ones, relative to the centre of gravity with a rotation-angle po. their lengths. Module F. The Z values for the markers are calculated from the Xfase and Yfase by: (2). 2, = X , . tan (Xfase) + Y , . tan (Yfase) (3) m refers to the marker positions P, Q, or R. Module G . From the spatial coordinates of the markers and the calculated tilt-angles the projections of the markers in the X-Y plane are calculated. First the perpendicular distance V, to the tilt-axis (Y-axis): (7) op",, and ORO, the measured vector lengths. The relative error is used to diminish the influence of the shape of the triangle on the resulting error. n is the number of tilt-angles and 520 R.JONGES ET AL. areas from all tiltrecordings area zeroreeording A Xfase V V calculate area in space calculate tilt angles > original vector lengths from all tiltrecordings D C > tilt angle corrections > tilt angle ~ balancing .- - - - - - - - - - - I I ? I I I I I vector length calculations I I I I I I I I I I I I I I -1 Fig. 3. The alignment procedure. There are three inputs. One in- record (input modules A and B). The third input is formed by the projected vector lengths of a marker (origin centre of gravity) meaput consists of the coordinates of three projected markers P ( x y , Q(xy,, , and R,,, derived from the zero-record (input module E). The second sured in each tilt-record (input module I). input concerns the areas of projected triangles P’Q’R’ in each tilt- Modules J and K. Both modules use the output of module I in an interactive way. Starting value for the Yfase is 0”. The starting value for the Xfuse depends on the area of the projected triangles. If the largest triangle area belongs to the zero-record than the starting value is 0”; otherwise the starting value is the angle calculated from the largest projected triangle area and that of the zero-record. The angles Xfase, Yfase, and the rotation-angle po are iteratively adjusted depending on the minimal value found by module I. Xfuse and Yfase are alternately changed by small angles. At a given Xfase, the Yfase is changed until the best fit is found. From that position the Xfuse is changed until the best fit and so on. At every change of the Xfase or Yfase the “basic rotation procedure tilt-axis” searches for the best fit of the rotation-angle po. Output of the procedure 1. The tilt-axis position in the zero-record, i.e., its rotation-angle po. 2. The spatial X, Y, and Z coordinates of the three markers P, Q, and R, where the origin of the coordinate system is in the centre of gravity and the Y-axis coincides with the tilt-axis. 3. The tilt-angles of the tilt-records. To align the micrographs properly, for every micrograph its rotation angle pi has to be derived from the calculated parameters. This is done, by calculating the coordinates of P, Q, and R at every tilt-angle (output of module G) and rotating the micrographs with the correct angles. The rotation angle per micrograph is determined by calculating the angle between calculated marker position and the measured marker position for each vector (centre of gravity to a projected marker). The three rotation angles for the projected markers P , &, and R should ideally be equal but because of errors they are not. Therefore, the rotation-angle of a micrograph is taken to be the average result of these three angles. Tilt-angle balancing. Balancing of the tilt-angles is performed in module D of the alignment procedure. It provides a connection with the tilt-angle stages of the E.M. as they were set. Tilt-series are taken with equal tilt-angle ranges on either side of the zero-record. The alignment procedure uses this equality (balanced 52 1 THREE-POINT REPOSITIONING OF AXES 0 tz , I 0 The alignment procedure calculates the tilt-angles (Oil by: I 0, = arccos OPb \ (8) with OPb the projection in the zero-record and OP; the projection a t the tilt-angle ei. Inset (a) of Figure 4 shows the calculated angles from equation 8 with the correct Xfase = X f l . If an intermediate incorrect plane orientation is generated by the alignment procedure (Xfase = X f 2 ) , the radius of arc (B) equals OP2. The projected vectors must have the same length as those on arc (A). The tilt positions of vector OP2 are drawn as small circles on arc (B). Figure 4b shows the tilt-angles resulting from equation 8 and Xfase = Xf2. Comparison of Figure 4a and b shows the effects of choosing a wrong Xfase. In Figure 4b the set of positive and negative angles is, I * obviously, a wrong set of angles. To prevent such a Fig. 4. Balancing of the tilt-angles. Arc A represents the trajec- derailment we developed the balancing principle. The tory of point P1 at the correct plane orientation angle Xfose = X f l . tilt-angles are adjusted with the average of the positive OP’ is the projection in the zero-recordof OP1.The small circles on arc and the negative tilt-angles. By giving all the tilt-anA are the tilt-positions of OP1.Arc B corresponds to a wrong Xfuse gles (including the zero-record) an offset angle in the value ( X f z ) ;the circles correspond to the tilt-positions of OP2 where the projections are equal to those on arc A. The insets (a and b) show correct direction, the intermediate position of the plane the tilt-angles of the tilt-positions on arcs A and B, respectively (ver- through the spatial markers is not in accordance with tical: tilt-angle zero degrees). the calculated tilt-angles anymore. Especially the calculated projections of the markers belonging to the zero-record receive a deviation from their true projections and force an error at the output of the alignment set of tilt-angles) to guide the alignment process to one procedure if the tilt-angles are not balanced. unique set of tilt-angles). The error between an interFor balancing it is not necessary to include all the mediate tilt-angle range (‘‘unbalanced)’)and its bal- tilt-angles. Any number of combinations with the same anced range is used to adjust the tilt-angles, thus forc- tilt (positive and negative) can be used, with a miniing an increase of the output error if they are not mum of one pair. A characteristic of the balancing prinbalanced. We called this “the balancing principle” of ciple is the freedom for every tilt-angle to settle in its the alignment procedure. A balanced tilt-angle range best fit in accordance with the area ratio of projected has a sum zero. Figure 4 illustrates the difference be- triangle and spatial triangle. tween a balanced and an unbalanced set of tilt-angles and the effect the Xfase has upon the calculated tiltAutomatic Centralizing and Detection angles. The tilt-axis coincides with the Y-axis, so the of the Markers Yfase has no effect on the area of the projected triangle, The advantage of gold particles is the identifiability therefore only the X-Z plane is shown and the area of the projected triangle is proportional to the projected of their projections which makes them useful for automatic detection of their centre points. Automatic cenvector length. Two situations for the Xfase are shown: the vector tre-point detection of the projection of a spherical gold OP1 having the correct angle X f 1 with the X-axis and particle has to cope with the projected material around a vector OP2 having a mis-oriented plane angle X f 2 . the projected disc. At least some contrast between the For both situations the vector OP‘ is the projection of marker and its surrounding should be present. ChangOP1 as well as of OP2 and its length is in accordance ing the tilt-angle results in changes of the projected surrounding densities and a decrease of the contrast at with the measured area in the zero-record. Assume that, starting from the correct situation large tilt-angles. To minimize the influence of sur( O P l ) ,the zero-record and twelve different records are rounding and change of contrast, we chose a detection made, six with negative and six with positive tilt-an- of the centre coordinates by a cross-correlation search gles a t intervals of 10”. On arc (A) the positions of procedure. The cross-correlation is performed between OP1 are drawn as small circles. The projected vector a centralized marker model, which has been created lengths are their perpendicular distances to the Z-axis. beforehand, and the object markers in the micrographs. The Xfase is controlled by the alignment procedure and This marker model is computed from a marker, manis yet an unknown parameter. To calculate the tilt- ually chosen within a circular domain of which the angles, thirteen projected distances of OP1 are in- diameter is approximately two times the marker diameter. The computer optimizes the circular model dovolved. I 522 R.JONGES ET AL. I (a) (b) (C) Fig. 5. Automatic centralization of a marker in a circular model space. a: Model domain with an eccentric marker. b: Domain (a) mirrored at line k and averaged. c: Domain (b) mirrored at line 1 and averaged. Result (c) is used for the detection in the micrograph of the coordinates of the original marker (a) by cross-correlationsearching. The procedure is repeated until the centralized marker model results in d. main in such a way that the centre of the model domain becomes the centre of the marker. Figure 5 shows the optimalization steps for creating a centralized marker model. 1. Manually a marker is chosen and a marker domain is indicated by the outer circle of Figure 5a. The white portion of the domain contains densities surrounding this particular marker. 2. A mirror image of the domain with respect to line k is calculated and the densities of both images are averaged (Fig. 5b). 3. Repeat step 2 for line 1. 4. By means of cross-correlation searching in the micrograph, the domain of Figure 5c is used as a model to locate the coordinates of the original marker we started with. 5. If there is a difference between the position located in step 4 and the original position, the steps 2, 3, and 4 are repeated, starting with the coordinates located at step 4 instead of step 1. 6. The iteration procedure is finished if at step 5 the newly found coordinates are equal to the latter set of coordinates. The result is a centralized marker model (Fig. 5d). This procedure has the advantage that it produces a model consisting of a centralized marker and an averaged surrounding. Involving more markers by adding these to the model domain improves the signal to noise ratio. After creation of an averaged marker model, it is used to search in every micrograph for the (object) markers, the coordinates of which are needed for the alignment procedure. The search areas in the micrographs were restricted manually to detect only the coordinates of the chosen markers. Material The alignment procedure was tested on rat testicular material, fixed in glutaraldehyde (2.5%)and postfixed in 1% OsO4 solution, dehydrated and embedded in Epon. Sections (100-200 nm) were cut on an ultramicrotome with a diamond knife (Reichart Ultracut E). The sections were collected on formvar coated slotgrids and contrasted with uranylacetate sometimes followed by a lead citrate. For alignment purposes, colloidal gold (10 nm) was applied before and after collecting the sections after lightly coating the surfaces with carbon. Sections were studied with an E.M.operating at 100 kV and relevant details were recorded at a magnification of x 15.000 or 18.000 (Philips EM420) using the “stereo conditions” of the lens program, which keeps the tilt-axis parallel with the Y-axis of the negative. Tilt series were taken from t-60” to -60” with increments of 5”. The micrographs were digitized with a flatbed scanner (Epson GT-6000). A pixel equals 1 nm2 and the grey level range was 8 bit. The alignment calculations and visualization of results were carried out on a workstation (IBM R6000 type 7012/32H). RESULTS Using two approaches we first investigated the detection accuracy of the cross-correlation searching technique with simulated data. In the first approach the correlation technique was tested to judge its sensitivity to changes in diameter of the markers. To that end a black marker was put into a white field contaminated with black pixels, randomly distributed. Zero noise refers t o a pure white surrounding of the marker; a 100%noise to a complete black surrounding where the marker cannot be recognized a t all. In the following, the term “object marker” is used for a marker being processed in a surrounding and the term “marker model” refers to a centralized marker in a circular domain. The latter one is used as the model which is correlated with an object marker. Three sizes of object markers were investigated, namely the standard size of ten pixel diameter (corresponding to the true marker size of 10 nm) and two sizes where diameters were either reduced or enlarged, by 10%.To locate the position of the object markers, a marker model of twenty pixels diameter (white) containing a centralized marker of ten pixels diameter (black) was set up. This marker model was refined by dividing each pixel into 10 x 10 subpixels. This made it possible to detect distances between the imposed positions and the detected ones as values from a discrete set of pixel lengths: 0.0, 0.1, 0.141, 0.2, etc. The cross-correlation searching was performed on the object marker within a radius of 50 pixels around it. The surrounding of the object marker was contaminated with black pixel noise at increments of 5%black filling. This was repeated 25 times for each object diameter. The surroundings of the three investigated object markers were each contaminated with the same sequence of white noise. Figure 6 shows the obtained accuracy (averaged) as a function of the black filling percentage for the three object marker sizes. Up to 75% black pixel filling the deviation error, i.e., the accuracy, is better than 1 pixel. Changes of 10%in size of the object marker have little influence. In the second approach, in order to quantify the deviation errors, we used an E.M. micrograph without any marker (Fig. 7a). This micrograph was digitized with a pixel of 1 nm’. As marker model we used the same model as described before. Next, we mixed a single object marker of 10 pixels in diameter with the micrograph. The marker had a radius of 5 pixels and was created on a discrete 11 x 11 pixel matrix. The middle pixel was the centre of the black disc and the THREE-POINT REPOSITIONING OF AXES -- 5.M 4.15 - I 4-50 4.25 2 .g object marker 4.w- ----_ 9 3.75- ---- ,a 3.50- -10 11 3.25- ;3w- 0) 2.15 - 2.50- '5 4 2.252.02- 1.75- f g 0 IS01.25- 1w- * 0.75 0.50 - o.w, 0 _ _ - -*-. , . , . , . , . , . , . , . *-___- 10 20 30 40 50 60 70 I 80 . , 90 . I 1W random black pixel filling (percentage) Fig. 6. Result of the test of the marker searching procedure. A marker model of 20 pixels is used. The object markers are black and have a diameter of 9, 10,and 11 pixels, respectively. The white surrounding of the object markers is contaminated with black pixels, randomly distributed. Horizontally, the percentage of black filling: 0 means a white and 100 a totally black surrounding. Vertically, the deviation error, i.e., the averaged distance (N = 25) between the correct marker position and the detected one. pixels on the rim received a portion of the disc depending on the distance to the centre. Mixing of this object marker with the micrograph was performed by taking the lowest grey value of the micrograph or the marker. In this way the micrograph grey values could dominate the rim of the marker. The object marker was located by cross-correlation searching, comparing the marker model with the micrograph in an area of 50 pixels around the mixed object marker. The measurement was repeated having the single object marker at every pixel coordinate of the micrograph successively. Figure 8 shows the histogram of the deviation errors. More than 95% of the measurements stay within 0.5 pixel and the average error is 0.16 pixel. The larger errors are found near the black areas of the micrograph (Fig. 7b). In this picture the densities represent the local deviation errors. White means an error zero and black represent the largest error found. The deviations are considered to represent the errors of the marker detection method as it was applied to the E.M. micrograph series. To assess the accuracy of the alignment method itself, theoretically precise projected marker positions were corrupted by values randomly taken from the continuous Gaussian distribution that fitted the histogram of Figure 8. The marker positions were the projections of the vertices of an equilateral triangle that was parallel to the projection plane in the zero-record. Projections were calculated for 25 tilt-angles from -60" 523 to +60" a t intervals of 5". The alignment procedure was repeated fifteen times, each time corrupting the projections with new random values. The deviation errors are depicted in Figure 9 as a function of the tiltangles, each vertex having a different symbol. The error is shown as a distance to the straight line over which the marker should move when tilting the specimen. The deviation errors vary randomly a t both sides of the straight lines. At the zero tilt-angle the match is exact because the zero-record was used as the reference. This figure shows that the alignment procedure keeps the deviations homogeneously distributed over all the projections. All deviations are within 0.5 pixel. In addition to the alignment, using the theoretical equilateral triangle, fifteen different series of 25 recorded E.M. micrographs each were investigated. Their deviation errors are represented in Figure 10. The deviation errors in both Figures 9 and 10 are comparable because the area of the triangle used for the test of the measuring errors (Fig. 9) was set to be equal to the average of the areas of the measured marker triangles that were used for the alignment of fifteen true E.M. tilt series of different specimen. The areas of the used triangles, measured in the zero-recordings, comprised approximately 110,000 pixels. The deviation errors in both figures present a quantitative representation of the alignment results using the same scaling. The results in both figures are comparable with the movement error, observed when an aligned micrograph series is presented sequentially as a movie. It is obvious that in the case of the micrograph series, the overall deviations are larger than the theoretical case (merely testing the measuring errors) of Figure 9. Moreover, the deviations increase towards the large tilt-angles. From Figure 9 it can be concluded that the results of the alignment procedure are very accurate and that other causes must be responsible for the differences between Figures 10 and 9. In addition we tested the alignment results by taking non-overlapping triangles chosen from the four quadrants of the same E.M. micrograph series. In these cases we observed, after alignment, a small increase in the deviation errors as the measuring errors had a stronger influence on the calculations due t o the relatively small areas of the triangles used. Comparing the resulting tilt-angles in the four quadrants we recorded that all the tilt-angles stayed within ~ 0 . 2 from " the average value. The direction of the local planes through the three markers expressed by Xfase and Yfase sometimes differs by more than 1.5". In general, we found that while the goniometer table of the E.M. is in its zero position, the slope of the surface of the specimen could have a deviation angle between 0" and 5" with respect to the projection plane. DISCUSSION This new procedure offers an integral solution for the alignment of E.M. tilt series, requiring a minimum of only three markers. The procedure always converges to a solution which has all the involved projections of the markers as close as possible to straight lines. The minimal required number of micrographs is three (including the zero-record),while there is no limit to the max- 524 R.JONGES ET AL. Fig. 7. a: E.M. micrograph of rat synaptonemal complex, sized 350 by 350 pixels without markers. On every pixel coordinate a n object marker has been mixed and has been subsequently located. b Presentation of the deviation errors of the marker localization procedure imum number. The deviation errors after alignment are directly related to the accuracy of the input coordinates of the three projected markers. Using a model marker in combination with the cross-correlation searching technique is a powerful prescription to allocate coordinates to the projected markers accurately in circumstances where the densities of the surrounding of the markers vary strongly. We distinguished two types of errors. First, the measuring errors due to the detection technique of the marker coordinates in the micrographs. Secondly, the deformation of the projections possibly caused by changes in the specimen during the registration of the micrographs, either through mechanical inaccuracy of the goniometer stage or by the optical system of the electron microscope. Deformation errors of the projections may be caused by various mechanisms which are mainly non-linear. If one does not know the particular sources of the error it is not easy to correct these non-linearities afterwards. On the other hand, the measuring error, i.e., the detection error of the projected marker coordinates, can be improved by using more markers. For the described alignment procedure three markers suffice, but there are no restrictions on using more markers. One could create artificial markers from the projections of any number of true markers in the following way. A group of markers is combined into an artificial marker by defining its centre of gravity in all the micrographs. The centres of gravity of three groups of markers form the input of the alignment procedure. In projection the three artificial in the micrograph. The densities of the pixels represent the local deviation error: white is a deviation error of 0 pixels and black the largest deviation error (in this case 0.82 pixel). 3ww -1 0.0 deviation error (pixel) Fig. 8. Deviation error distribution histogram measured in the micrograph of Figure 7 based upon 122,500 test positions. The averaged deviation error is 0.16 pixel and the maximal error is 0.82 pixel. For 95%of the marker localizations the deviation error remains below 0.5 pixel. THREE-POINT REPOSITIONING OF AXES v h 4.& a ('1 0.0 .... .... 4.5 -1.0: -70 . .60, . -50, . , . , . I . , 4 .M -m -10 1 , . o I 10 . I m T , M . 40, . M I . I 60 . , 70 tilt angle (degrees) . ' . * . -2.5 * ._ 1' . 3 . 0 ! . , . , . 1 . , . , . ~ . , . , . , . , . , . , , , . 1 -70 -60 -50 40 -30 -20 -10 0 10 20 M 40 M 6O 70 tilt angle (degrees) Fig. 9. Alignment accuracy of 15 measurements of theoretically precise projection coordinates corrupted by values randomly taken from the continuous Gaussian distributionof Figure 8. The deviation error is shown over the 25 projections for three markers (differently marked) as a distance to straight lines, perpendicular to the tilt-axis, over which the projected markers ideally should move. Fig. 10. The alignment result of 15 true E.M.micrograph tilt series (15 different specimens). The deviation errors are presented in the same way as in Figure 9 using the same scaling. This figure and Figure 9 quantitatively represent the alignment results. It is comparable with the movement error, observed when an aligned micrograph series is presented sequentially as a movie. markers behave like true markers. In cases where the alignment of tilt series must be carried out without the use of gold particles and the projection coordinates of specific points originating from the object have to be marked by hand in the micrographs, creating three artificial marker coordinates is a good alternative to improve the measuring accuracy. The test results after alignment, depicted in Figure 9, show the deviation errors related only to the marker detection accuracy. The errors are homogeneously dis- 525 tributed over all the micrographs with respect to straight lines perpendicular to the tilt-axis, over which the projections of the markers ideally move when tilting the specimen. The results of the alignment of true E.M. tilt series (Fig. 10)show that the deviation errors increase towards larger tilt-angles. This effect is most likely due to the non-linear effects of the E.M.imaging system in case of large tilt-angles. It is known that the microscope transfer function is highly defocus dependent (Hawkes, 1992). Sometimes this effect occurs at large negative tilt-angles, sometimes at large positive angles and sometimes at both. Therefore, depending on the size of the observed area and the thickness of the specimen, the error is probably due to an incorrect height adjustment of the microscope, resulting in an offset of the specimen in relation to the centre of the parallel electron-beam region. During the time that the micrographs are recorded the specimen may shrink. Our micrographs were recorded only after the specimen was stabilized in the E.M. (30-45 minutes). In some alignments we observed a slight gradual increase of the deviation error over the entire tilt range, possibly caused by extra shrinking of the specimen. For most of our series the error lies within the measuring accuracy of the markers. We do not favour linear scaling in the alignment procedure. Because of the mainly non-linear distortions of the records it is indistinct how such a scaling to the micrographs should be applied. An important part of the alignment procedure is the tilt-angle balancing principle. Using an equal number of positive and negative tilt-angles makes it possible to calculate the individual tilt-angles according to the area ratio, yet the whole tilt-angle range is comparable with the tilt-angle range taken by the electron microscope. The intervals between subsequent tilt-angles can change in any way on condition that for every positive tilt-record there is a negative tilt-record with the same tilt. When using the whole range of tilt-angles for the alignment procedure, the equality of the positive and negative angle is not critical. The balancing is carried out around the zero-record as the reference, since we expect the lowest errors of the position of the markers in this case. When using the tilt-angles as set on the E.M.(reading the nonius of the goniometer stage) as an input for the alignment procedure instead of the calculated tilt-angles from the cosine relation between areas, the deviation error increases a t those records where the measured area ratio is not in accordance with the E.M. tilt-angle ratio. As a result of the alignment procedure the spatial coordinates of three markers with respect to their centre of gravity are known. This means that, if one of the markers was chosen on the opposite side of the specimen, approximately the thickness of the specimen is known. The thickness of the specimen can also be obtained by putting the aligned micrographs in a movie loop. Thickness estimation is then performed by observing the moving surroundings related to artificial markings that are projected in the micrographs at calculated depths from the tilt-axis. The thickness is an important parameter for the reconstruction of the vol- 526 R. JONGES ET AL. Frank, J., McEwen, B., Radermacher, M., Turner, J., and Rieder, C. (1987)Three-dimensional tomographic reconstruction in high voltage electron microscopy. J. Electron Microsc. Tech., 6:193-205. Guckenberger, R. (1982)Determination of a common origin in the micrographs of tilt series in three dimensional electron microscopy. . Ultramicroscopy, 9167-174. Hawkes, P. (1992)The electron microscope as a structure projector. In: Electron Microscope. Frank, ed. Plenum Press, New York, pp. 17ACKNOWLEDGMENTS 38. The authors gratefully acknowledge Prof. Dr. J. f n g , Z.,and Sachs, F. (1991)Alignment of tomographic projections using an incomplete set of fiducial markers. Ultramicroscopy, 3 5 Strackee for his helpful comments and suggestions. 37-43. They are thankful to Dr. K. Jonges for his general as- Lawrence, M. (1983)Alignment of images for three-dimensional resistance and the excellent technical assistance of A. construction of non-periodic objects. Roc. Electron Microsc. Soc. S. Vink and H. van Veen. As a partner in a “Joint Study Afr., 13:19-20. Agreement,” IBM The Netherlands is especially ac- Lawrence, M. (1992)Least-squares method of alignment using markers. In: Electron Microscope.J . Frank, ed. Plenum Press, New York, knowledged for placing an IBM R6000 workstation pp. 197-204. type 7012/32H at our disposal which greatly enhanced Luther, P., Lawrence, M., and Crowther, R. (1988)A method for monour computing facilities. itoring the collapse of plastic sections as a function of electron dose. Ultramicroscopy, 247-18. REFERENCES Marle, J.v., Dietrich, A,, Jonges, K., Jonges, R., de Moor, E., Vink, A,, Boon, P., and van Veen, H. (1995)Em-tomomaDhv of section colBerriman, J., Bryan, R., Freeman, R., and Leonard, K. (1984)Methlapse, a non linear phenomenon. Microsc. Reg. Tech., 31:311-316. ods for specimen thickness determination in electron microscopy. Olins, D., Olins, A., Levy, H., Durfee, R., S.M., Margle, Tinnel, E., and Ultramicroscopy, 13:351-364. Dover, S. (1983)Electron microscopy tomography: Transcription in Bonnet, N., Quintana, C., Favard, P., and Favard, N. (1985)Threethree dimensions. Science, 220498-500. dimensional graphical reconstruction from hvem stereoviews of bi- Olins, A., Olins, D., Olman, V., Levy, H., D.P., and BazettJones ological specimens by means of a microcomputer. Biol. Cell, 55: (1994)Modelling the 3-d rna distribution in the balbiani ring gran125-138. ule. Chromosoma, 103:302-310. Farrow, N., and Ottensmeyer, F. (1993)Automatic 3d alignment of projection images of randomly oriented objects. Ultramicroscopy, 52:141-156. ume. The thickness restricts the volume into which the density information of the micrographs has to be materialized. Altogether we have shown that three markers are suMicient for an accurate alignment.