More Mosaic MadnessHomographyRotational Mosaics3D → 2D Perspective Projection3D Rotation ModelPairwise alignmentRotation about vertical axisDo we have to project onto a plane?Full PanoramasCylindrical projectionCylindrical ProjectionInverse Cylindrical projectionCylindrical panoramasCylindrical image stitchingAssembling the panoramaProblem: DriftFull-view (360°) panoramasSpherical projectionSpherical ProjectionInverse Spherical projection3D rotationFull-view PanoramaPolar ProjectionOther projections are possibleCylindrical reprojectionWhat’s your focal length, buddy?DistortionRadial distortionMore Mosaic Madness15-463: Computational PhotographyAlexei Efros, CMU, Fall 2006© Jeffrey Martin (jeffrey-martin.com)with a lot of slides stolen from Steve Seitz and Rick SzeliskiHomographyA: Projective – mapping between any two PPs with the same center of projection•rectangle should map to arbitrary quadrilateral •parallel lines aren’t•but must preserve straight lines•same as: project, rotate, reproject called HomographyPP2PP11yx*********wwy'wx'H pp’ To apply a homography H•Compute p’ = Hp (regular matrix multiply)•Convert p’ from homogeneous to image coordinatesRotational MosaicsCan we say something more about rotational mosaics?i.e. can we further constrain our H?3D → 2D Perspective Projectionu(Xc,Yc,Zc)ucfK3D Rotation ModelProjection equations1. Project from image to 3D ray(x0,y0,z0) = (u0-uc,v0-vc,f)2. Rotate the ray by camera motion(x1,y1,z1) = R01 (x0,y0,z0)3. Project back into new (source) image(u1,v1) = (fx1/z1+uc,fy1/z1+vc)Therefore: Our homography has only 3,4 or 5 DOF, depending if focal length is known, same, or different.•This makes image registration much better behaved(u,v,f)(x,y,z)f(x,y,z)(u,v,f)R11010KRKHPairwise alignmentProcrustes Algorithm [Golub & VanLoan]Given two sets of matching points, compute Rpi’ = R pi with 3D rayspi = N(xi,yi,zi) = N(ui-uc,vi-vc,f)A = Σi pi pi’T = Σi pi piT RT = U S VT = (U S UT) RTVT = UT RTR = V UTRotation about vertical axisWhat if our camera rotates on a tripod?What’s the structure of H?mosaic PPDo we have to project onto a plane?Full PanoramasWhat if you want a 360 field of view?mosaic Projection Cylinder•Map 3D point (X,Y,Z) onto cylinderCylindrical projectionXYZunit cylinderunwrapped cylinder•Convert to cylindrical coordinatescylindrical image•Convert to cylindrical image coordinatesCylindrical ProjectionYXInverse Cylindrical projectionXYZ(X,Y,Z)(sin,h,cos)Cylindrical panoramasSteps•Reproject each image onto a cylinder•Blend •Output the resulting mosaicCylindrical image stitchingWhat if you don’t know the camera rotation?•Solve for the camera rotations–Note that a rotation of the camera is a translation of the cylinder!Assembling the panoramaStitch pairs together, blend, then cropProblem: DriftVertical Error accumulation•small (vertical) errors accumulate over time•apply correction so that sum = 0 (for 360° pan.)Horizontal Error accumulation•can reuse first/last image to find the right panorama radiusFull-view (360°) panoramasSpherical projectionunwrapped sphere•Convert to spherical coordinatesspherical image•Convert to spherical image coordinatesXYZ),,(1)ˆ,ˆ,ˆ(222ZYXZYXzyx•Map 3D point (X,Y,Z) onto sphere)ˆ,ˆ,ˆcoscossincos(sin zyxSpherical ProjectionYXInverse Spherical projectionXYZ(x,y,z)(sinθcosφ,cosθcosφ,sinφ)cos φφcos θ cos φsin φ3D rotationRotate image before placing on unrolled sphereXYZ(x,y,z)(sinθcosφ,cosθcosφ,sinφ)cos φφcos θ cos φsin φ_ __ _p = R pFull-view Panorama++++++++Polar ProjectionExtreme “bending” in ultra-wide fields of viewOther projections are possibleYou can stitch on the plane and then warp the resulting panorama•What’s the limitation here? Or, you can use these as stitching surfaces •But there is a catch…f = 180 (pixels)Cylindrical reprojectionf = 380f = 280Image 384x300top-down viewFocal length – the dirty secret…What’s your focal length, buddy?Focal length is (highly!) camera dependant•Can get a rough estimate by measuring FOV:•Can use the EXIF data tag (might not give the right thing)•Can use several images together and try to find f that would make them match•Can use a known 3D object and its projection to solve for f•Etc.There are other camera parameters too:•Optical center, non-square pixels, lens distortion, etc.DistortionRadial distortion of the image•Caused by imperfect lenses•Deviations are most noticeable for rays that pass through the edge of the lensNo distortion Pin cushion BarrelRadial distortionCorrect for “bending” in wide field of view lensesUse this instead of normal
View Full Document