DOC PREVIEW
CMU CS 15463 - Homographies and mosaic

This preview shows page 1-2-3-25-26-27 out of 27 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 27 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Homographies and MosaicsWhy Mosaic?Why Mosaic?Why Mosaic?Mosaics: stitching images togetherHow to do it?A pencil of rays contains all viewsAligning imagesImage reprojectionImage reprojectionBack to Image WarpingHomographyImage warping with homographiesImage rectificationLeast Squares ExampleSolving for homographiesFun with homographiesPanoramaschanging camera centerPlanar scene (or far away)Planar mosaicProgramming Project #3Bells and WhistlesBells and WhistlesBells and WhistlesFrom Last Year’s classGo Explore!Homographies and Mosaics15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005© Jeffrey Martin (jeffrey-martin.com)with a lot of slides stolen fromSteve Seitz and Rick SzeliskiWhy Mosaic?Are you getting the whole picture?• Compact Camera FOV = 50 x 35°Slide from Brown & LoweWhy Mosaic?Are you getting the whole picture?• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°Slide from Brown & LoweWhy Mosaic?Are you getting the whole picture?• Compact Camera FOV = 50 x 35°• Human FOV = 200 x 135°• Panoramic Mosaic = 360 x 180°Slide from Brown & LoweMosaics: stitching images togethervirtual wide-angle cameraHow to do it?Basic Procedure• Take a sequence of images from the same position– Rotate the camera about its optical center• Compute transformation between second image and first• Transform the second image to overlap with the first• Blend the two together to create a mosaic• If there are more images, repeat…but wait, why should this work at all?• What about the 3D geometry of the scene?• Why aren’t we using it?A pencil of rays contains all viewsrealcamerasyntheticcameraCan generate any synthetic camera viewas long as it has the same center of projection!Aligning imagesTranslations are not enough to align the imagesleft on top right on topmosaic PPImage reprojectionThe mosaic has a natural interpretation in 3D• The images are reprojected onto a common plane• The mosaic is formed on this plane•Mosaic is a synthetic wide-angle cameraImage reprojectionBasic question• How to relate two images from the same camera center?– how to map a pixel from PP1 to PP2PP2PP1Answer• Cast a ray through each pixel in PP1• Draw the pixel where that ray intersects PP2But don’t we need to know the geometryof the two planes in respect to the eye?Observation:Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to anotherBack to Image WarpingTranslation2 unknownsAffine6 unknownsPerspective8 unknownsWhich t-form is the right one for warping PP1 into PP2?e.g. translation, Euclidean, affine, projectiveHomographyA: Projective – mapping between any two PPs with the same center of projection• rectangle should map to arbitrary quadrilateral • parallel lines aren’t• but must preserve straight lines• same as: project, rotate, reprojectcalled HomographyPP2PP1⎥⎥⎦⎤⎢⎢⎣⎡⎥⎥⎦⎤⎢⎢⎣⎡=⎥⎥⎦⎤⎢⎢⎣⎡1yx*********wwy'wx'Hpp’To apply a homography H• Compute p’ = Hp (regular matrix multiply)•Convert p’ from homogeneous to image coordinatesImage warping with homographiesimage plane in front image plane belowblack areawhere no pixelmaps toImage rectificationTo unwarp (rectify) an image• Find the homography H given a set of p and p’ pairs• How many correspondences are needed?• Tricky to write H analytically, but we can solvefor it!• Find such H that “best” transforms points p into p’• Use least-squares!pp’Least Squares ExampleSay we have a set of data points (X1,X1’), (X2,X2’), (X3,X3’), etc. (e.g. person’s height vs. weight)We want a nice compact formula (a line) to predict X’s from Xs: Xa + b = X’We want to find a and bHow many (X,X’) pairs do we need?What if the data is noisy?'22'11XbaXXbaX=+=+⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡⎥⎦⎤⎢⎣⎡'2'12111XXbaXXAx=B⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎣⎡=⎥⎦⎤⎢⎣⎡⎥⎥⎥⎥⎦⎤⎢⎢⎢⎢⎣⎡.........111'3'2'1321XXXbaXXXoverconstrained2min BAx −Solving for homographiesCan set scale factor i=1. So, there are 8 unkowns.Set up a system of linear equations:Ah = bwhere vector of unknowns h = [a,b,c,d,e,f,g,h]TNeed at least 8 eqs, but the more the better…Solve for h. If overconstrained, solve using least-squares: Can be done in Matlab using “\” command• see “help lmdivide”⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡=⎥⎥⎥⎦⎤⎢⎢⎢⎣⎡1yxihgfedcbawwy'wx'p’ = Hp2min bAh −Fun with homographiesSt.Petersburgphoto by A. TikhonovVirtual camera rotationsOriginal imagePanoramas1. Pick one image (red)2. Warp the other images towards it (usually, one by one)3. blendchanging camera centerDoes it still work?synthetic PPPP1PP2Planar scene (or far away)PP3 is a projection plane of both centers of projection, so we are OK!This is how big aerial photographs are madePP1PP3PP2Planar mosaicProgramming Project #3Homographies and Panoramic Mosaics• Capture photographs (and possibly video)• can check out cameras and/or tripods from me (1 day loan)• Compute homographies (define correspondences)• will need to figure out how to setup system of eqs.• (un)warp an image (undo perspective distortion)• Produce 3 panoramic mosaics (with blending)• Do some of the Bells and WhistlesBells and WhistlesBlending and Compositing• use homographies to combine images or video and images together in an interesting (fun) way. E.g.– put fake graffiti on buildings or chalk drawings on the ground– replace a road sign with your own poster– project a movie onto a building wall–etc.Bells and WhistlesCapture creative/cool/bizzare panoramas• Example from UW (by Brett Allen):• Ever wondered what is happening inside your fridge while you are not looking? Capture a 360 panorama (next class)Bells and WhistlesVideo Panorama• Capture two (or more) stationary videos (either from the same point, or of a planar/far-away scene). Compute homography and produce a video mosaic. Need to worry about synchronization (not too hard). • e.g. capturing a football game from the sides of the stadiumOther interesting ideas?•talk to meFrom Last Year’s classEunjeong Ryu (E.J), 2004 Ben Hollis, 2004 Ben Hollis, 2004 Matt Pucevich, 2004Go Explore!Ken Chu,


View Full Document

CMU CS 15463 - Homographies and mosaic

Documents in this Course
Lecture

Lecture

36 pages

Lecture

Lecture

31 pages

Wrap Up

Wrap Up

5 pages

morphing

morphing

16 pages

stereo

stereo

57 pages

mosaic

mosaic

32 pages

faces

faces

33 pages

MatTrans

MatTrans

21 pages

matting

matting

27 pages

matting

matting

27 pages

wrap up

wrap up

10 pages

Lecture

Lecture

27 pages

Lecture

Lecture

40 pages

15RANSAC

15RANSAC

54 pages

lecture

lecture

48 pages

Lecture

Lecture

42 pages

Lecture

Lecture

11 pages

Lecture

Lecture

52 pages

Lecture

Lecture

39 pages

stereo

stereo

57 pages

Lecture

Lecture

75 pages

texture

texture

50 pages

Lectures

Lectures

52 pages

Load more
Download Homographies and mosaic
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Homographies and mosaic and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Homographies and mosaic 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?