Multi-perspective PanoramasPictures capture memoriesPanoramasBad panorama?No geometrically consistent solutionScientists solution to panoramas: Single center of projectionFrom sphere to planeDistorted panoramasObjectivesStand on the shoulders of giantsCartographic projectionsCommon panorama projectionsGlobal ProjectionsLearn from the artistsTwo horizons!Renaissance painters solutionPersonalized projectionsMultiple planes of projectionSlide 19Slide 20Slide 21Applying personalized projectionsSlide 23Slide 24Objectives - revisitedMulti-view compositionsWhy multi-view?Multi-view panoramasLong ImagingSmooth Multi-ViewWhat’s wrong in the picture?Non-smoothThe ChairJoiners are popularMain goals: Automate joiners Generalize panoramas to general image collectionsSlide 36Slide 37Slide 38What’s going on here?A cacti gardenPrinciplesSlide 42Slide 43Slide 44Slide 45AlgorithmStep 1: Feature matchingStep 2: AlignStep 3: OrderOrdering imagesSlide 51Slide 52Slide 53Can we do better?Step 4: Improve alignmentIterate Align-Order-ImportanceIterative refinementSlide 58Slide 59What is this?That’s me readingAnza-BorregoTractorArt reproductionSlide 65Slide 66Manual by PhotographerOur automatic resultFailure?GUIThe Impossible BridgeHomage to David HockneyTake homeThank You15-463 Class Project from 2007Multi-perspective PanoramasSlides from a talk by Lihi Zelnik-Manor at ICCV’07 3DRR workshopPictures capture memoriesPanoramasRegistration: Brown & Lowe, ICCV’05Blending: Burt & Adelson, Trans. Graphics,1983Visualization: Kopf et al., SIGGRAPH, 2007Bad panorama?Output of Brown & Lowe softwareNo geometrically consistent solutionScientists solution to panoramas:Single center of projectionRegistration: Brown & Lowe, ICCV’05Blending: Burt & Adelson, Trans. Graphics,1983Visualization: Kopf et al., SIGGRAPH, 2007No 3D!!!From sphere to planeDistortions are unavoidableDistorted panoramasOutput of Brown & Lowe softwareActual appearanceObjectives1. Better looking panoramas 2. Let the camera move:•Any view•Natural photographingStand on the shoulders of giantsCartographersArtistsCartographic projectionsCommon panorama projectionsθφCylindircalPerspective StereographicGlobal ProjectionsCylindircalPerspective StereographicLearn from the artistsMultiple view pointsDe Chirico “Mystery and Melancholy of a Street”, 1914perspectiveperspectiveSharp discontinuityTwo horizons!Renaissance painters solution“School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!Personalized projections“School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!Multiple planes of projection Sharp discontinuities can often be well hiddenOur multi-view resultSingle viewOur multi-view resultSingle viewOur multi-view resultSingle viewApplying personalized projectionsForegroundInput imagesBackgroundpanoramaSingle viewOur multi-view resultSingle viewOur multi-view resultObjectives - revisited1. Better looking panoramas 2. Let the camera move:•Any view•Natural photographingMultiple views can live togetherMulti-view compositionsDavid Hockney, Place Furstenberg, (1985)3D!!Melissa Slemin, Place Furstenberg, 2003Why multi-view?Multiple viewpoints Single viewpointDavid Hockney, Place Furstenberg, 1985Multi-view panoramasSingle view MultiviewRequires video inputZomet et al. (PAMI’03)Long ImagingAgarwala et al. (SIGGRAPH 2006)Smooth Multi-ViewGoogle mapsWhat’s wrong in the picture?Google mapsNon-smoothGoogle mapsThe ChairDavid Hockney (1985)Joiners are popular4,985 photos matching joiners. 4,007 photos matching Hockney.41 groups about HockneyThousands of membersFlickr statistics (Aug’07):Main goals: Automate joiners Generalize panoramas to general image collectionsObjectives•For Artists:Reduce manual laborManual: ~40min.Fully automaticObjectives•For Artists:Reduce manual labor•For non-artists:Generate pleasing-to-the-eye joinersObjectives•For Artists:Reduce manual labor•For non-artists:Generate pleasing-to-the-eye joiners•For data exploration:Organize images spatiallyWhat’s going on here?A cacti gardenPrinciplesPrinciples•Convey topologyCorrectIncorrectPrinciples•Convey topology•A 2D layering of imagesBlending: blurryGraph-cut: cuts hoodDesired joinerPrinciples•Convey topology•A 2D layering of images•Don’t distort imagesrotate scaletranslatePrinciples•Convey topology•A 2D layering of images•Don’t distort images•Minimize inconsistenciesGoodBadAlgorithmStep 1: Feature matchingBrown & Lowe, ICCV’03Step 2: AlignLarge inconsistenciesBrown & Lowe, ICCV’03Step 3: OrderReduced inconsistenciesOrdering imagesTry all orders: only for small datasetsOrdering imagesTry all orders: only for small datasetscomplexity: (m+n)m = # imagesn = # overlaps = # acyclic ordersOrdering imagesObservations:–Typically each image overlaps with only a few others–Many decisions can be taken locallyOrdering imagesApproximate solution: –Solve for each image independently–Iterate over all imagesCan we do better?Step 4: Improve alignmentIterate Align-Order-ImportanceIterative refinementInitial FinalIterative refinementInitial FinalIterative refinementInitial FinalWhat is this?That’s me readingAnza-BorregoTractorPaolo Uccello, 1436Art reproductionPaolo Uccello, 1436Zelnik & Perona, 2006Art reproductionSingle view-pointZelnik & Perona, 2006Art reproductionManual by PhotographerOur automatic resultFailure?GUIThe Impossible BridgeHomage to David Hockney•Incorrect geometries are possible and fun!•Geometry is not enough, we need scene analysis•A highly related work:"Scene Collages and Flexible Camera Arrays,”Y. Nomura, L. Zhang and S.K. Nayar,Eurographics Symposium on Rendering, Jun, 2007.Take homeThank You15-463 Class Project from
View Full Document