CSCE 641 Computer Graphics:CSCE 641 Computer Graphics: Image MosaicingJinxiang ChaiSource:faculty.cs.tamu.edu/jchai/cpsc641_spring10/lectures/lecture8.pptOutlineOutlineImage registrationImage registration- How to break assumptions? 3D-2D registrationImage mosaicingggMosaics: Stitching Image TogetherMosaics: Stitching Image Togethervirtual wide-angle cameraMosaic ProcedureMosaic ProcedureBasic ProcedureMosaic ProcedureMosaic ProcedureBasic ProcedureTake a sequence of images from the same–Take a sequence of images from the same position• Rotate the camera about its optical centerMosaic ProcedureMosaic ProcedureBasic ProcedureTake a sequence of images from the same–Take a sequence of images from the same position• Rotate the camera about its optical center– Compute transformation between second image and firstMosaic ProcedureMosaic ProcedureBasic ProcedureTake a sequence of images from the same–Take a sequence of images from the same position• Rotate the camera about its optical center– Compute transformation between second image and firstTransform the second image to overlap with the–Transform the second image to overlap with the firstMosaic ProcedureMosaic ProcedureBasic ProcedureTake a sequence of images from the same–Take a sequence of images from the same position• Rotate the camera about its optical center– Compute transformation between second image and firstTransform the second image to overlap with the–Transform the second image to overlap with the first–Blend the two together to create a mosaicgMosaic ProcedureMosaic ProcedureBasic ProcedureTake a sequence of images from the same–Take a sequence of images from the same position• Rotate the camera about its optical center– Compute transformation between second image and firstTransform the second image to overlap with the–Transform the second image to overlap with the first–Blend the two together to create a mosaicg– If there are more images, repeatImage MosaicImage MosaicIs a pencil of rays contains all viewsrealcamerasyntheticcameraCan generate any synthetic camera viewas long as it has the same center of projection!Image Re-projectionImage Reprojectionmosaic PPThe mosaic has a natural interpretation in 3D–The images are reprojected onto a common planeThe images are reprojected onto a common plane– The mosaic is formed on this plane– Mosaic is a synthetic wide-angle cameraIssues in Image MosaicIssues in Image MosaicHow to relate two images from the same camera center?- image registrationHow to re-project images to a common plane?image warping-image warping??mosaic PPImage MosaicingImage MosaicingGeometric relationship between imagesGeometric relationship between imagesImage MosaicingImage MosaicingGeometric relationship between imagesGeometric relationship between images– Use 8-parameter projective transformation matrix' xcbax111' yhgfedyImage MosaicingImage MosaicingGeometric relationship between imagesGeometric relationship between images– Use 8-parameter projective transformation matrix' xcbax111' yhgfedy– Use a 3D rotation model (one R per image)0200100rfrr121012112012011100200100frffrfrrfrrrfrrDerive it by yourself!Image MosaicingImage MosaicingGeometric relationship between images– Use 8-parameter projective transformation matrix– Use a 3D rotation model (one R per image)Register all pairwise overlapping images–Feature-based registrationFeaturebased registration– Pixel-based registrationChain together inter-frame rotationsImage StitchingImage StitchingStitch pairs together, blend, then cropImage StitchingImage StitchingA big image stitched from 5 small imagesgg gPanoramasPanoramasWhat if you want a 360 field of view?mosaic Projection CylinderCylindrical PanoramasCylindrical PanoramasSteps–Re-project each image onto a cylinder– Blend –Output the resulting mosaicCylindrical ProjectionCylindrical Projection– Map 3D point (X,Y,Z) onto li dcylinderYXYZunit cylinderCylindrical ProjectionCylindrical Projection– Map 3D point (X,Y,Z) onto li dcylinderY–Convert to cylindrical coordinatesXYZConvert to cylindrical coordinatesunit cylinderunwrapped cylinderunwrapped cylinderCylindrical ProjectionCylindrical Projection– Map 3D point (X,Y,Z) onto li dcylinderY–Convert to cylindrical coordinatesXYZConvert to cylindrical coordinates– Convert to cylindrical image coordinatescoordinates• s defines size of the final imageunit cylinderunwrapped cylinderunwrapped cylindercylindrical imageCylindrical PanoramasCylindrical PanoramasABBmosaic Projection CylinderCannot map point A to Point B without knowing (X,Y,Z)Cylindrical PanoramasCylindrical PanoramasBABCmosaic Projection CylinderBut we can map point C (images) to Point B.pp ( g )Cylindrical WarpingCylindrical WarpingGiven focal length fdi tand image center (xc,yc)Y(X,Y,Z)(sin,h,cos)XYZCylindrical PanoramasCylindrical PanoramasMap image to cylindrical or spherical coordinates–need known focal lengthf = 180 (pixels)Image 384x300 f = 380f = 280Cylindrical PanoramaCylindrical Panorama3D rotation registration of four images taken with a handheld camera.Cylindrical PanoramaCylindrical PanoramaRecognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemItIInput ImagesOutput panorama #1Output panorama #1Recognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemItIInput ImagesOutput panorama #2Recognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemItIInput ImagesOutput panorama #3Recognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemInput Images- How to recognize which images can be used for panoramas?How to stitch them automatically?-How to stitch them automatically?Recognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemRecognizing panoramasRecognizing panoramas•A fully automatic 2D image stitcher systemA fully automatic 2D image stitcher systemImage matching withSIFTfeatures-Image matching with SIFTfeatures- For every image, find the M best images with RANSAC- Form a graph and find connected