A Survey of Image Registration TechniquesLISA GOTTESFELD BROWNDepartment of Computer Sctence, Colunzbza Unzl,ersity, New York, NY 10027Registration M a fundamental task in image processing used to match two or morepictures taken, for example, at different times, from different sensors, or from differentviewpoints. Virtually all large systems which evaluate images require the registrationof images, or a closely related operation, as an intermediate step. Specific examples ofsystems where image registration is a significant component include matching a targetwith a real-time image of a scenefor target recognition, monitoring global land usageusing satellite images, matching stereo images to recover shape for autonomousnavigation, and aligning images from different medical modalities for diagnosis.Over the years, a broad range of techniques has been developed for various types ofdata and problems. These techniques have been independently studied for severaldifferent applications, resulting in a large body of research. This paper organizes thismaterial by estabhshing the relationship between the variations in the images and thetype of registration techniques which can most appropriately be applied. Three majortypes of variations are distinguished. The first type are the variations due to thedifferences in acquisition which cause the images to be misaligned. To register images,a spatial transformation is found which will remove these variations. The class oftransformations which must be searched to find the optimal transformation isdetermined by knowledge about the variations of this type. The transformation class inturn influences the general technique that should be taken. The second type ofvariations are those which are also due to differences in acquisition, but cannot bemodeled easily such as lighting and atmospheric conditions. This type usually effectsintensity values, but they may also be spatial, such as perspective distortions, Thethn-d type of variations are differences in the images that are of interest such as objectmovements, growths, or other scene changes. Variations of the second and third typeare not directly removed by registration, but they make registration more difficultsince an exact match is no longer possible. In particular, it is critical that variations ofthe third type are not removed. Knowledge about the characteristics of each type ofvariation effect the choice of feature space, similarity measure, search space, andsearch strategy which will make up the final technique. All registration techniques canbe viewed as different combinations of these choices. This framework M useful forunderstanding the merits and relationships between the wide variety of existingtechniques and for assisting in the selection of the most suitable Iechnique for aspecific problem.Categories and Subject Descriptors: A. 1[General Literature]: Introductory andSurvey; 1.2.10[Artificial Intelligence]: Vision and SceneUnderstanding; 1.4[Computing Methodologies]: Image Processing; 1.5 [Computing Methodologies]:Pattern RecognitionGeneral Terms: Algorithms, Design, Measurement, PerformanceAdditional Key Words and Phrases: Image registration, image warping, rectification,template matchingPermission to copy without fee all o. part of this material is granted provided that the copies are not madeor distributed for direct commercial advantage, the ACM copyright notice and the title of the publicationand Its data appear, and notie is given that copying is by permission of the Associatlon for ComputingMachinery. To copy otherwise, or to repubhsh, requires a fee and/or specific permission.@ 1992 ACM 0360-0300/92/’ 1200-0325 $01.50ACM Comput,ng Surveys,VoI 24, No. 4, December1992326 “ Lisa G. Brown1CONTENTS1. INTRODUCTION2 IMAGE REGISTRATION IN THEORY2 1 Defimtlon22 Tran~f~=matlon23 Image Varlatlons24 Rectlticatlon3 REGISTRATION METHODS31 Correlatmn and Sequential Methods32 Fourier Methods33 Point Mapping34 E1.ast,cModel-BasedMatch]ng35 Summary4 CHARACTERISTICS OF REGISTRATIONMETHODS41 Feature Space42 SlmdarIty Measure43 SearchSpaceand Strategy44Summary~—INTRODUCTIONA frequent problem arises when imagestaken, at different times, by differentsensors or from different viewpoints needto be compared. The images need to bealigned with one another so that differ-ences can be detected. A similar problemoccurs when searching fora. prototype ortemplate in another image. To find theoptimal match for the template in theimage, the proper alignment between theimage and template must be found. All ofthese problems, and many related varia-tions, are solved by methods that per-form image registration. A transforma-tion must be found so that the points inone image can be related to their corre-sponding points in the other. The deter-mination of the optimal transformationfor registration depends on the types ofvariations between the images. The ob-jective of this paper is to provide a frame-work for solving image registration tasksand to survey the classical approaches.Registration methods can be viewed asdifferent combinations of choices for thefollowing four components:(1) a feature space,(2) a search space,(3) a search strategy, and(4) a similarity metric.Thefeature space extracts the informa-tion in the images that will be used formatching. Thesearch space is the classof transformations that is capable ofaligning the images. Thesearch strategydecides how to choose the next transfor-mation from this space, to be tested inthe search for the optimal transforma-tion. Thesimilarity metric determinesthe relative merit for each test. Searchcontinues according to the search strat-egy until a transformation is found whosesimilarity measure is satisfactory. As weshall see, the types of variations presentin the images will determine the selec-tion for each of these components.For example, consider the problem ofregistering the two x-ray images of chesttaken of the same patient at differenttimes shown in Figure 1. Properly align-ing the two images is useful for detect-ing, locating, and measuring pathologicaland other physical changes. A standardapproach to registration for these imagesmight be as follows: the images mightfirst be reduced to binary images by de-tecting the edges or regions of highestcontrast using a standard edge detectionscheme. This removes extraneous infor-mation and reduces the amount of datato be evaluated. If it is thought that theprimary difference in acquisition of theimages was a small