Image registration methods: a surveyIntroductionImage registration methodologyFeature detectionArea-based methodsFeature-based methodsSummaryFeature matchingArea-based methodsFeature-based methodsTransform model estimationGlobal mapping modelsLocal mapping modelsMapping by radial basis functionsElastic registrationImage resampling and transformationEvaluation of the image registration accuracyCurrent trends and outlook for the futureAcknowledgementsReferencesImage registration methods: a surveyBarbara Zitova´*, Jan FlusserDepartment of Image Processing, Institute of Information Theory and Automation, Academy of Sciences of the Czech RepublicPod voda´renskou veˇzˇı´4, 182 08 Prague 8, Czech RepublicReceived 9 November 2001; received in revised form 20 June 2003; accepted 26 June 2003AbstractThis paper aims to present a review of recent as well as classic image registration methods. Image registration is the process of overlayingimages (two or more) of the same scene taken at different times, from different viewpoints, and/or by different sensors. The registrationgeometrically align two images (the reference and sensed images). The reviewed approaches are classified according to their nature (area-based and feature-based) and according to four basic steps of image registration procedure: feature detection, feature matching, mappingfunction design, and image transformation and resampling. Main contributions, advantages, and drawbacks of the methods are mentioned inthe paper. Problematic issues of image registration and outlook for the future research are discussed too. The major goal of the paper is toprovide a comprehensive reference source for the researchers involved in image registration, regardless of particular application areas.q 2003 Elsevier B.V. All rights reserved.Keywords: Image registration; Feature detection; Feature matching; Mapping function; Resampling1. IntroductionImage registration is the process of overlaying two ormore images of the same scene taken at different times,from different viewpoints, and/or by different sensors. Itgeometrically aligns two images—the reference andsensed images. The present differences between imagesare introduced due to different imaging conditions. Imageregistration is a crucial step in all image analysis tasksin which the final information is gained from thecombination of various data sources like in image fusion,change detection, and multichannel image restoration.Typically, registration is required in remote sensing(multispectral classification, environmental monitoring,change detection, image mosaicing, weather forecasting,creating super-resolution images, integrating informationinto geographic information systems (GIS)), in medicine(combining computer tomography (CT) and NMR datato obtain more complete information about the patient,monitoring tumor growth, treatment verification,comparison of the patient’s data with anatomical atlases),in cartography (map updating), and in computer vision(target localization, automatic quality control), to namea few.During the last decades, image acquisition devices haveundergone rapid development and growing amount anddiversity of obtained images invoked the research onautomatic image registration. A comprehensive survey ofimage registration methods was published in 1992 byBrown [26]. The intention of our article is to cover relevantapproaches introduced later and in this way map the currentdevelopment of registration techniques. According to thedatabase of the Institute of Scientific Information (ISI), inthe last 10 years more than 1000 papers were published onthe topic of image registration. Methods published before1992 that became classic or introduced key ideas, which arestill in use, are included as well to retain the continuity andto give complete view of image registration research. We donot contemplate to go into details of particular algorithms ordescribe results of comparative experiments, rather we wantto summarize main approaches and point out interestingparts of the registration methods.In Section 2 various aspects and problems of imageregistration will be discussed. Both area-based and feature-based approaches to feature selection are described inSection 3. Section 4 reviews the existing algorithms forfeature matching. Methods for mapping function design aregiven in Section 5. Finally, Section 6 surveys main0262-8856/03/$ - see front matter q 2003 Elsevier B.V. All rights reserved.doi:10.1016/S0262-8856(03)00137-9Image and Vision Computing 21 (2003) 977–1000www.elsevier.com/locate/imavis*Corresponding author. Tel.: þ 420-2-6605-2390; fax: þ 420-2-8468-0730.E-mail address: [email protected] (B. Zitova´), [email protected](J. Flusser)techniques for image transformation and resampling.Evaluation of the image registration accuracy is coveredin Section 7. Section 8 concludes main trends in the researchon registration methods and offers the outlook for the future.2. Image registration methodologyImage registration, as it was mentioned above, is widelyused in remote sensing, medical imaging, computer visionetc. In general, its applications can be divided into four maingroups according to the manner of the image acquisition:Different viewpoints (multiview analysis). Images of thesame scene are acquired from different viewpoints. The aimis to gain larger a 2D view or a 3D representation of thescanned scene.Examples of applications: Remote sensing—mosaicingof images of the surveyed area. Computer vision—shaperecovery (shape from stereo).Different times (multitemporal analysis). Images of thesame scene are acquired at different times, often on regularbasis, and possibly under different conditions. The aim is tofind and evaluate changes in the scene which appearedbetween the consecutive image acquisitions.Examples of applications: Remote sensing—monitoringof global land usage, landscape planning. Computervision—automatic change detection for security monitor-ing, motion tracking. Medical imaging—monitoring of thehealing therapy, monitoring of the tumor evolution.Different sensors (multimodal analysis). Images of thesame scene are acquired by different sensors. The aim is tointegrate the information obtained from different sourcestreams to gain more complex and detailed scenerepresentation.Examples of applications: Remote sensing—fusion ofinformation from sensors with different characteristics likepanchromatic images, offering better spatial