Project 2: Panoramic Mosaic StitchingComputer Vision – CS766 – Fall 2007Brandon M. SmithOutlineIntroductionApproachResultsList of Submitted FilesReferencesIntroductionSeveral algorithms were used and implemented for this project, including those for:A. Removing radial distortionB. Warping images onto cylindrical coordinatesC. Finding Scale Invariant Feature Transform (SIFT) pointsD. Matching these points between adjacent imagesE. Random Sample Consensus (RANSAC) for finding suitable homographiesF. Image transformationG. Image StitchingApproachAll of the “code” used for this project was written in MATLAB. This was convenient since MATLAB – the Image Processing Toolbox in particular – has many built-in functions that make some tasks easier. A good example of this is image transformation. MATLAB conveniently provides an imtransform function that nicely transforms images, with nice sampling and reconstruction (i.e., using bicubic interpolation, which is what I used).The main MATLAB script is mosaic.m and is located in the “program” directory. It is used like this:result_img = mosaic(filename, f, k1, k2);whereresult_img is the panoramic mosaic resultfilename is the name of a text file that contains a list of all of the image files that will go into the panorama. Brandon Smith Page 1 of 7 CS766 – Project 2f is the focal length. In our case, since we used the Canon A640 with tag 4726208885 at 480x640 resolution, f = 663.3665.k1 and k2 are radial distortion coefficients. In our case they were k1 = -0.19447 and k2 = 0.23382Three portions of this assignment were not written by me. I did not write the script for correcting radial distortion (see rect.m), getting an initial set of matching points between images (match.m), or the script for finding SIFT features (sift.m).For image stitching, I used a technique suggested by the TA, Yu-Chi. The amount of contribution by each original pixel to the final pixel at each location is determined by a weighted average. The weights are calculated based on the squared distance from each pixel to its corresponding image center. So, given img1 and img2, the resulting pixel would be( 1/r1^2 * p1 + 1/r2^2 * p2 ) / (1/r1^2 + 1/r1^2)where r1 and r2 are the distances from the center of img1 and img2 to p1 and p2 respectively, and p1 and p2 are pixel values from img1 and img2 respectively. ResultsChris Hinrichs and I took pictures at several locations in and around campus, including the center of Camp Randall Stadium, the hill in Camp Randall, a hallway in Engineering Hall, the Union (not shown), the fountain outside Memorial Library, and State Street (not shown). Those images are shown in Figures 1 – 3. Figure 1: Camp Randall Stadiuma. individual images b. panoramic mosaic imageBrandon Smith Page 2 of 7 CS766 – Project 2Figure 2: the hill in Camp Randalla. individual images b. panoramic mosaic imageFigure 3: inside Engineering Halla. individual images b. panoramic mosaic imageThe test image was used first to create a panorama. The result is shown in Figure 4. Notice the gradual slant down and to the right. The previous images had the same problem, but this was corrected and the images cropped using the GIMP image processing software. Brandon Smith Page 3 of 7 CS766 – Project 2Figure 4: uncropped test image panoramaFigure 5 shows an image taken outside of Memorial Library from the center of the found. It illustrates the effect of warping. Notice in the original image on the left the fountain ledge, which is obviously curved. However, if a panorama is to be made, this ledge should span the entire 360 degree image in a straight line. The warped image on the right shows this clearly. Figure 5: example of warpingThe results obtained were reasonably good. However, some blurring is visible, especially near the top and bottom of the panorama. This may be due, in part, to slight imperfections in the camera calibration, i.e., f and k1 and k1 might not be totally accurate. It may also be caused by non-ideal transformations, i.e., the homography chosen by the algorithm might not be the most ideal one. The images were generated with k = 50 and the threshold set to 2 pixels. Better results could probably be obtained by calculating an average of the best homographies discovered, or perhaps by setting k higher (the number of attempts at finding a homography), or by altering the threshold.Brandon Smith Page 4 of 7 CS766 – Project 2List of Submitted Filesimages/ (directory)hallway/ (directory)Contains a set of images taken in a hallway inside Engineering Hall.hill/ (directory)Contains a set of images taken on a hill in Camp Randall.results/ (directory)Contains the panoramic mosaic image results.hallway.jpgPanorama of inside Engineering Hall.hill.jpgPanorama of the hill in Camp Randallstadium.jpgThis is my favorite result. It is a panorama of the inside of Camp Randall Stadium. test_image.jpgPanorama generated from the test images.small2/ (directory)Contains all of the test images.stadium/ (directory)Contains a set of images taken inside Camp Randall Stadium.warp_example.jpgSee Figure 5 above. program/ (directory)appendimages.mdefs.hLICENSEMakefilematch.cmatch.mshowkeys.msift*sift.msiftWin32.exeBrandon Smith Page 5 of 7 CS766 – Project 2util.ctmp.keytmp.pgmUsed by the SIFT programapply_distortion.mrect.mTwo files taken from [4]. They are used to correct radial distortion.mosaic.mThe main program. See above for usage.calc_final_homography.mImplements RANSAC to find the homography that will “best” transform an image. composite_images.mPuts two images together, with dissolved edges.get_transformed_xy.mSimple function that returns a transformed version of x, y based on input x, y and a homography. pad_image.mReturns a padded version of an image, enlarged in the x- and y-directions by a specified amount. readimages.mReads in all of the images listed in a text file. solve_for_homography.mGiven a set of four matched points between two images, calculates the homography between them.warpimg.mUses rect.m to correct radial distortion, then warps the image based on the focal length.hallway_images.txthill_images.txtstadium_images.txttest_images.txtContain a list of image file names to be used to create a panoramic mosaic. These files can be read in by mosaic.mREADME.txtBrandon Smith Page 6 of 7 CS766 – Project 2Describes very briefly how to run the mosaic program.report.htmlreport.pdfreport.docYou’re reading one of these.References[1]