imagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...1 of 7 10/15/2009 12:26 AMPanoramic Image MosaicChaman Singh Verma and Mon-JuContents:Introduction1.Theory and Implementation Details2.Cylindrical Image Warping1.SIFT Feature Detections2.Ransac Translation.3.Post Processing:3.Image Blending1.Feathering Algorithm.1.Image Pyramid Algorithm.2.Drift Correction.2.Performance and Results4.Software usage.5.References.6.Introduction:Generating full view panoramic images is important for both commercial and artistic value. Since the inception ofphotography many specific devices have been invented to create panoramic images. With the availability ofinexpensive digital camera, the desire to create full panoramic images is overwhelming and importance ofautomatic image stitching is quite high. In this project, we create panoramic images using Cylindrical Warping. Cylindrical warping is easiest to implement,but it has stringent requirement that all the images must be taken with level camera or with the known tiltedangle. With this method, full homography calculations are not needed only the translation along the angulardirection is required to create the panoramic images. Forward Warping:From image coordinates (x,y), the projected coordinates on the cylinder (x', y') are given by :Inverse Warping:inverse mapping from cylindrical coordinates (x',y') to the image (x,y) is :imagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...2 of 7 10/15/2009 12:26 AMIn forward warping the source image is mapped onto cylindrical surface, but it can holes in the destination image ( because some pixels may never get mapped there). therefore we use inverse mapping where each pixel in the destination image is mapped to the source image. Since the mapping is unlikely to be exactly on the pixel values, bilinear interpolation is used to calculate the colors at the destination pixels.Radial Distortion:Because of the thick lens that are often used in the camera, it is necessary to correct the radial distortions in theimage. One of the simplified distortion model that is commonly used is Xd = Xu( 1 + k1*r^2 + k2*r^4) Yd = Yu( 1 + k1*r^2 + k2*r^4 )Where (Xd,Yd) are the distorted image position and (Xu,Yu) are the undistorted correct position. The values (k1,k2) that depends on the camera can be calibrated using some standard techniques.Both inverse mapping and radial distortion correction requires interpolation to calculate the color values at the destination pixel.o Interpolation is not only expensive, they also smooth the features, therefore, this step must be minimized to get high quality final images. In this application there are two places where interpolated values are required (1) Cylindrical to image in inverse warping (2) From undistorted image values to distorted images values. We can combine these two steps and calculate cylindrical image values directly from the distorted images and therefore, avoid the intermediate interpolation values.SIFT Feature detectionWe directly use SIFT algorithm to produce features in every image. Each SIFT descriptor is 128 char long. These features are matched with neighboring image to estimate the translation. Since there could be small number of outliers that has potential to misalign the final images. To find out these outliers, we use RANSAC algorithm.imagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...3 of 7 10/15/2009 12:26 AMRansac TranslationRansac algorithm is general purpose algorithm that can be used to calculate full homography in the presense of outliers. The use of cylindrical warping has the advantage that only the translation motion has to calculated on the warped image. Also for the translation estimation, only one feature is sufficient. Ransac estimation counts the inliners based on some tolerance value ( d < eps) which depends on the noise present in the images. Since our image capturing was high quality, only two pixel tolerance was sufficient. ( We found that that in our images only 5-10% outliers).Image Blending:When different images are stitched together, for various reasons (changed lighting conditions, vignette effects) the adjacent pixel intensities differ enough to produce artifacts as shown in the following pictures. To remove these artifacts, we experimented with two algorithms (1) Feathering (2) Image pyramids. Feathering or center weighting image blending:In this simplest approach, the the pixel values in the blended regions are weighted average from the two overlapping images. Sometimes this simple approach doesn't work( for example in the presence of exposure differences ). But in our case, all the images were taken at the same time and using high quality tripods, therefore, this simple algorithm produces excellant results. PB(i,j) = (1-w)*PA(i,j) + w*PB(i,j)imagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...4 of 7 10/15/2009 12:26 AM Pyramid Blening:Laplacian pyramid is an algorithm using Gaussian to blend the image while keeping the significant feature in themean time. It downsizes the image into different levels (sizes) with Gaussian. Later it expands the Gaussian in tothe lower lever and subtracts from the image in that lever to acquire the Laplacian image.After generating Laplacian pyramids for the overlap images A and B, we combine the two images in differentLaplacian levels by combining partial images from each of them.imagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...5 of 7 10/15/2009 12:26 AMAfterward, we expand the LS from the top level () to the next level (N-1) and add it to the original Laplacian image in the corresponding layer () to generate the latest Laplacian image in the corresponding layer (). We repeat this step until reaching ground level () and the final result will be the blending imageimagemosaic file:///home/csverma/public/html/CS766_09/ImageMosaic/imagemosaic...6 of 7 10/15/2009 12:26 AM Apple (Image A) Orange ( Image B)Feathering Blend Laplace Pyramid BlendDrift Correction:Very often the first image and the last image in the view don't align properly. This misalignment can be adjusted by shear warping and other bundle adjustment algorithm. In our case, we didn't have to do any adjustment as the misaligment was only 1-2 pixels.Results:We used Canon SX100 and