1Eliminating Ghosting and Exposure Artifacts in Image MosaicsMatthew UyttendaeleMicrosoft ResearchAshley EdenHarvard UniversityRichard SzeliskiMicrosoft ResearchAbstractAs panoramic photography becomes increasinglypopular, there is a greater need for high-qualitysoftware to automatically create panoramic images.Existing algorithms either produce a rough "stitch" thatcannot deal with common artifacts, or require userinput. This paper presents methods for dealing with twoartifacts that often occur in practice. Our firstcontribution is a method for dealing with objects thatmove between different views of a dynamic scene. Ifsuch moving objects are left in, they will appear blurryand "ghosted". Treating such regions as nodes in agraph, we use a vertex cover algorithm to selectivelyremove all but one instance of each object. Our secondcontribution is a method for continuously adjustingexposure across multiple images in order to eliminatevisible shifts in brightness or hue. We computeexposure corrections on a block-by block basis, thensmoothly interpolate the parameters using a spline toget spatially continuous exposure adjustment. Ourenhancements, combined with previously publishedtechniques for automatic image stitching, result in ahigh-quality automated stitcher that exhibits far fewerartifacts than existing software.1. IntroductionAs panoramic photography becomes increasinglypopular, there is a greater need for software to createpanoramic images. Ideally, the image stitching processshould be completely automatic, requiring no userinformation in calculating the panorama [1,10,11,12].This not only applies to registering the images, but alsoto fixing irregularities typical to amateur photography.Two such irregularities discussed in this paper aremovement of objects within a scene, and differences inexposure between images.One of the problems in automatic image stitching is thatof de-ghosting. When the images are taken, there is noguarantee that objects in the image stay stationary fromone image to the next. This becomes a problem in theareas of overlap between images. When the images arestitched together, we take a composite of theoverlapping images in order to create a smooth transitionbetween neighbors. However, if regions of the scene arenot stationary, the overlap image will be slightlydifferent in each image contributing to the overlap.Thus, those regions of the composited image willcontain combinations of pixel values from entirelydifferent parts of the scene. For example, if a personmoves his head in an area of overlap, the regioncontaining his head in the stitched image will be acombination of head and background from both images.It will give the head a ghosted look, not to mention thatthis ghosted head will appear in two locations.In order to get around this problem of ghosting, we needto display the stitched image as if nothing in the scenemoved. Thus, when regions of the scene do havemovement, we would like to use pixel values from onlyone of the contributing images for that region. In orderto accomplish this, however, we need to determine a)where the movement occurred, and b) which image touse.Several methods have been proposed to eliminateghosts. Shum and Szeliski [10] propose a method fordeghosting small misregistrations based on computingoptic flow and then doing a multi-way morph. Medianfilters are often used and are effective when more thanhalf the images contain consistent pixels [3, 9]. This isnot the case for mosaics created from a relatively smallnumber of images. Davis [2] proposes cutting imagesbetween regions of movement, finding the best cut withDijkstra’s algorithm. However, it is not clear how togeneralize this to mosaics created from manyoverlapping images. We need a novel algorithm totackle the more complicated problem of multipleoverlapping regions of movement.Another problem in automatic image stitching isexposure differences between images. Exposuredifferences are a common occurrence, especially withdigital photographs. If the differences are not corrected,the panorama will appear to have seams, even when theimages are blended in overlapping regions. Additionallyour difference-based de-ghosting algorithm could2interpret exposure differences as moving objects. Thus,we must find a way to equalize the exposure of eachimage based on the information in neighboring imageswhile retaining local smoothness.Previous work in this area uses a large number of imagesof the same scene to do a radiometric calibration of thecamera [4,5,6]. In our work we don’t assume acalibration step. In work by Hasler et al [7], the imageregistration and the camera’s parametric Opto-ElectronicConversion Function are simultaneously computed forpairs of images. This assumes a known parametricmodel for the camera and it is not clear how togeneralize this to multiple overlapping images. Burtand Adelson [13] use multi-resolution splines to performspatial blending between different images. However,since their method depends on band-pass imagepyramids, it is not clear how to apply it to the irregularlyshaped images present in general image mosaics.Furthermore, current stitching techniques [1,10] alreadyuse large "feathering" regions, so multi-resolutionsplining may not help. In our work we computecorrections on a block-by-block basis, and thensmoothly interpolate the parameters using a spline to getspatially continuous exposure adjustment.2. Ghosting Artifacts2.1 Where Does Movement Occur?The first step in our de-ghosting process is to determinewhich regions in the input images are not static and thusdiffer across images. We limit the search for regions ofdifference (ROD) to the areas of overlap between inputimages. To identify RODs, a map is computed for eachinput image by flagging pixels which differ by more thana certain threshold from pixels in overlapping images.To smooth the difference maps, a morphological erodeand dilate step is then applied. Next, a region extractionalgorithm is applied to identify and label contiguousregions. Figure 1 depicts the construction of differencemaps for a simple mosaic.Our overall goal is to use information from only oneimage for each ROD. Thus, we must groupcorresponding regions across images, keep informationfrom one of the images, and ignore correspondinginformation in the other images. But how do we groupcorresponding regions? Not all corresponding regionsare the same size. Because more than one image mayoverlap with a given image, and each