Unformatted text preview:

6.098/6.882 Computational Photography 1Problem Set 5Assigned: April 13, 200 6Due: April 27, 2006Problem 1 Panorama ImagingIn this problem we will guide you to build panorama by yourself. Youare not allowed to use any other software for this problem set. We ask youto generate intermediate results at each step. You are also NOT allowed touse MATLAB functions such as cp2tform, tformarray, tformfwd, tforminv,maketform or imtransform.Step 1. Take picturesTake at least four pictures that you will stitch into a panorama. You mayborrow a digital camera and tripod from Ce (32-D46 0). Here are some hintsfor taking the pictures.• Make sure images overlap by ∼ 40%.• Keep the same viewpoint.• Use a tripod.• See with distinct features (not repetitive).• Try to keep the same exposure (use manual mode if available on yourcamera, but check aperture and shudder speed to make sure t he theimage lo oks nature).• Avoid lenses with too much distortion (fish eye and other lenses thatdo not preserve straight lines). Avoid the wide-angle range of zoomsbecause they tend to have such distortion.• Down-sample the images to be smaller than 800x600 before processing.Debug with even lower resolution, e.g. 400x300.6.098/6.882 Computational Photography 2• No moving objects in the scene f or the first trial. You may shoot movingobjects and try to display them properly later for extra credits.Step 2. Mark correspondencesWhile it is possible to create a panorama generation program that is en-tirely automatic, using feature detectors and descriptors to find correspon-dences, in order to simplify the problem, we will skip tha t step and allow youto mark correspondences by hand.Select the center picture as the reference, and mark correspondence be-tween each image and the reference. If it is impossible to mark correspon-dence to the reference, mark to the nearest and t hen you will propagate thecorrespondence in a later step. For this case you may want to build a treestructure of the images, where the reference is the r oot and each picture cancorrespond to the reference picture by tracking the tree to the root.To mark correspondence between a pair of images, you can use your favoritetool. One suggestion is to use MATLAB function subplot to display the twoimages side by side, and ginput to click the feature points. You may clickone feature point at image 1, and then click the corresponding feature pointat image 2, and so on. At the returning matrix from ginput, the odd rowswill be the feature points at image 1 and the even rows be the correspondingfeature points at image 2.Step 3. Solve for homographyWrite code to compute the homography between two images g iven pairsof correspondences. Setup a set of linear equations as explained in lectureto relate the 8 coefficients of the homography matrix to the correspondencepairs. Recall t hat the ninth coefficient (bottom-right) can be set to 1. Usethe matlab command \ to find a least-square solution. Display the 3x3homography transform matrix for each pair. Both transforms and inversetransforms are needed for generating panorama.(Extra credit): refine the correspondences. For each pair, search in the localneighborhood of o ne image (e.g. 5x5 pixels) for the point that minimizes theSSD (sum of square differences) of the pixels in a small window (5x5 too )with the ot her image. If you are hard core, use sub-pixel accuracy.Step 4. Compute the coordinate of the panorama6.098/6.882 Computational Photography 3To compose the final panorama we need to generate the coordinate systemfor the big panorama picture that can embrace all the images warped to thereference frame. You may warp the corners of each picture to the coo r dinateof the reference frame, and then choose the minimum and maximum of thecorners to determine the frame of the panorama. Use MATLAB functionline to display t he frame of each picture in the panorama in blue, and thereference frame in red.Step 5. WarpSince you already know the corners o f each frame in the panorama, you canuse MATLAB function inpolygon to generate a binary mask for each frameto approximately indicate the pixels that will show up from that frame atthe panorama. Then apply the homography transform (before doing thatyou need to translate the coordinate back to the reference frame) and useinterp2 to generate the warped image. Be careful with the pixels that areoutside the input picture. Modify the binary mask accordingly. You alsoneed to generate the mask and the translated image for the reference frame,though you don’t have to go through this process. Display the masks andthe warped frames.Step 6. BlendYou now have n images at the final resolution and n corresponding binarymasks. You need to blend them together.First, implement a na¨ıve solution where you use the binary mask to blendthe pictures in an arbitrar y order (e.g. from left to r ig ht). Start from anempty image and for each warped picture, overwrite the pixels where themask is 1. You will probably have artifacts at the boundary, but it will allowyou to verify that the images are a ctually aligned.(6.882 only) In order to blend the panorama seamless, we want to usepyramid blending as seen in problem set 2 . We need to compute appropriatemasks and deal with the fact t hat we have more than two images to blend.Our solution will be based on distance maps that indicate how fa r each pixelis from the boundary of the input images. You can make your life easyand use a Manhattan distance where the distance to the boundary simplybecomes min(x, y, width − x, height − y). Use the same warp procedure togenerate the distance map for each picture in the final frame (aligned withthe reference picture). Put value 0 for pixels outside the picture. Now for6.098/6.882 Computational Photography 4each pixel, we have n values that store the distances to the boundary of eachinput.(a) Implement simple blending where, for each pixel, you use the color ofthe input image with the biggest distance to the boundary. Instead ofbeing at the boundary like in the previous blending, artif acts will nowoccur at pixels equidistant from the boundaries.(b) Implement a smoother version where you use for each pixel the averageof the n inputs colors weighted by distance. Do not forget to divideby the sum of the distance. The transition should be smoother, butghosting might occur.(Extra credit) Generalize the above two distance-based blending to multi-scale pyramid blending as in problem set 2.Bells and


View Full Document

MIT 6 098 - Problem Set 5

Download Problem Set 5
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Problem Set 5 and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Problem Set 5 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?