Image and Vision Computing 21 2003 977 1000 www elsevier com locate imavis Image registration methods a survey Barbara Zitova Jan Flusser Department of Image Processing Institute of Information Theory and Automation Academy of Sciences of the Czech Republic Pod voda renskou ve z 4 182 08 Prague 8 Czech Republic Received 9 November 2001 received in revised form 20 June 2003 accepted 26 June 2003 Abstract This paper aims to present a review of recent as well as classic image registration methods Image registration is the process of overlaying images two or more of the same scene taken at different times from different viewpoints and or by different sensors The registration geometrically align two images the reference and sensed images The reviewed approaches are classified according to their nature areabased and feature based and according to four basic steps of image registration procedure feature detection feature matching mapping function design and image transformation and resampling Main contributions advantages and drawbacks of the methods are mentioned in the paper Problematic issues of image registration and outlook for the future research are discussed too The major goal of the paper is to provide a comprehensive reference source for the researchers involved in image registration regardless of particular application areas q 2003 Elsevier B V All rights reserved Keywords Image registration Feature detection Feature matching Mapping function Resampling 1 Introduction Image registration is the process of overlaying two or more images of the same scene taken at different times from different viewpoints and or by different sensors It geometrically aligns two images the reference and sensed images The present differences between images are introduced due to different imaging conditions Image registration is a crucial step in all image analysis tasks in which the final information is gained from the combination of various data sources like in image fusion change detection and multichannel image restoration Typically registration is required in remote sensing multispectral classification environmental monitoring change detection image mosaicing weather forecasting creating super resolution images integrating information into geographic information systems GIS in medicine combining computer tomography CT and NMR data to obtain more complete information about the patient monitoring tumor growth treatment verification comparison of the patient s data with anatomical atlases in cartography map updating and in computer vision Corresponding author Tel 420 2 6605 2390 fax 420 2 84680730 E mail address zitova utia cas cz B Zitova flusser utia cas cz J Flusser 0262 8856 03 see front matter q 2003 Elsevier B V All rights reserved doi 10 1016 S0262 8856 03 00137 9 target localization automatic quality control to name a few During the last decades image acquisition devices have undergone rapid development and growing amount and diversity of obtained images invoked the research on automatic image registration A comprehensive survey of image registration methods was published in 1992 by Brown 26 The intention of our article is to cover relevant approaches introduced later and in this way map the current development of registration techniques According to the database of the Institute of Scientific Information ISI in the last 10 years more than 1000 papers were published on the topic of image registration Methods published before 1992 that became classic or introduced key ideas which are still in use are included as well to retain the continuity and to give complete view of image registration research We do not contemplate to go into details of particular algorithms or describe results of comparative experiments rather we want to summarize main approaches and point out interesting parts of the registration methods In Section 2 various aspects and problems of image registration will be discussed Both area based and featurebased approaches to feature selection are described in Section 3 Section 4 reviews the existing algorithms for feature matching Methods for mapping function design are given in Section 5 Finally Section 6 surveys main 978 B Zitova J Flusser Image and Vision Computing 21 2003 977 1000 techniques for image transformation and resampling Evaluation of the image registration accuracy is covered in Section 7 Section 8 concludes main trends in the research on registration methods and offers the outlook for the future 2 Image registration methodology Image registration as it was mentioned above is widely used in remote sensing medical imaging computer vision etc In general its applications can be divided into four main groups according to the manner of the image acquisition Different viewpoints multiview analysis Images of the same scene are acquired from different viewpoints The aim is to gain larger a 2D view or a 3D representation of the scanned scene Examples of applications Remote sensing mosaicing of images of the surveyed area Computer vision shape recovery shape from stereo Different times multitemporal analysis Images of the same scene are acquired at different times often on regular basis and possibly under different conditions The aim is to find and evaluate changes in the scene which appeared between the consecutive image acquisitions Examples of applications Remote sensing monitoring of global land usage landscape planning Computer vision automatic change detection for security monitoring motion tracking Medical imaging monitoring of the healing therapy monitoring of the tumor evolution Different sensors multimodal analysis Images of the same scene are acquired by different sensors The aim is to integrate the information obtained from different source streams to gain more complex and detailed scene representation Examples of applications Remote sensing fusion of information from sensors with different characteristics like panchromatic images offering better spatial resolution color multispectral images with better spectral resolution or radar images independent of cloud cover and solar illumination Medical imaging combination of sensors recording the anatomical body structure like magnetic resonance image MRI ultrasound or CT with sensors monitoring functional and metabolic body activities like positron emission tomography PET single photon emission computed tomography SPECT or magnetic resonance spectroscopy MRS Results can be applied for instance in radiotherapy and