Multi-perspective PanoramasSlides from a talk by Lihi Zelnik-Manor at ICCV’07 3DRR workshopPictures capture memoriesPanoramasRegistration: Brown & Lowe, ICCV’05Blending: Burt & Adelson, Trans. Graphics,1983Visualization: Kopf et al., SIGGRAPH, 2007Bad panorama?Output of Brown & Lowe softwareNo geometrically consistent solutionScientists solution to panoramas:Single center of projectionRegistration: Brown & Lowe, ICCV’05Blending: Burt & Adelson, Trans. Graphics,1983Visualization: Kopf et al., SIGGRAPH, 2007No 3D!!!From sphere to planeDistortions are unavoidableDistorted panoramasOutput of Brown & Lowe softwareActual appearanceObjectives1. Better looking panoramas2. Let the camera move:• Any view• Natural photographingStand on the shoulders of giantsCartographersArtistsCartographic projectionsCommon panorama projectionsθφCylindircalPerspective StereographicGlobal ProjectionsCylindircalPerspective StereographicLearn from the artistsMultiple view pointsDe Chirico “Mystery and Melancholy of a Street”, 1914perspectiveperspectiveSharp discontinuityTwo horizons!Renaissance painters solution“School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!Personalized projections“School of Athens”, Raffaello Sanzio ~1510 Give a separate treatment to different parts of the scene!!Multiple planes of projectionSharp discontinuities can often be well hiddenOur multi-view resultSingle viewOur multi-view resultSingle viewOur multi-view resultSingle viewApplying personalized projectionsForegroundInput imagesBackgroundpanoramaSingle viewOur multi-view resultSingle viewOur multi-view resultObjectives - revisited1. Better looking panoramas2. Let the camera move:• Any view• Natural photographingMultiple views can live togetherMulti-view compositionsDavid Hockney, Place Furstenberg, (1985)Melissa Slemin, Place Furstenberg, 2003Why multi-view?Multiple viewpoints Single viewpointDavid Hockney, Place Furstenberg, 1985Multi-view panoramasSingle view MultiviewRequires video inputZomet et al. (PAMI’03)Long ImagingAgarwala et al. (SIGGRAPH 2006)Smooth Multi-ViewGoogle mapsWhat’s wrong in the picture?Google mapsNon-smoothGoogle mapsThe ChairDavid Hockney (1985)Joiners are popular4,985 photos matching joiners. 4,007 photos matching Hockney.41 groups about HockneyThousands of membersFlickr statistics (Aug’07):Main goals: Automate joinersGeneralize panoramas to general image collectionsObjectives• For Artists:Reduce manual laborManual: ~40min.Fully automaticObjectives• For Artists:Reduce manual labor• For non-artists:Generate pleasing-to-the-eye joinersObjectives• For Artists:Reduce manual labor• For non-artists:Generate pleasing-to-the-eye joiners• For data exploration:Organize images spatiallyWhat’s going on here?A cacti gardenPrinciplesPrinciples• Convey topologyCorrectIncorrectPrinciples• Convey topology• A 2D layering of imagesBlending:blurryGraph-cut:cuts hoodDesired joinerPrinciples• Convey topology• A 2D layering of images• Don’t distort imagesrotate scaletranslatePrinciples• Convey topology• A 2D layering of images• Don’t distort images• Minimize inconsistenciesGoodBadAlgorithmStep 1: Feature matchingBrown & Lowe, ICCV’03Step 2: AlignLarge inconsistenciesBrown & Lowe, ICCV’03Step 3: OrderReduced inconsistenciesOrdering imagesTry all orders: only for small datasetsOrdering imagesTry all orders: only for small datasetscomplexity: (m+n)m = # imagesn = # overlaps= # acyclic ordersOrdering imagesObservations:– Typically each image overlaps with only a few others– Many decisions can be taken locallyOrdering imagesApproximate solution: – Solve for each image independently– Iterate over all imagesCan we do better?Step 4: Improve alignmentIterate Align-Order-ImportanceIterative refinementInitial FinalIterative refinementInitial FinalIterative refinementInitial FinalWhat is this?That’s me readingAnza-BorregoTractorPaolo Uccello, 1436Art reproductionPaolo Uccello, 1436Zelnik & Perona, 2006Art reproductionSingle view-pointZelnik & Perona, 2006Art reproductionManual by PhotographerOur automatic resultFailure?GUIThe Impossible BridgeHomage to David Hockney• Incorrect geometries are possible and fun!• Geometry is not enough, we need scene analysis• A highly related work:"Scene Collages and Flexible Camera Arrays,”Y. Nomura, L. Zhang and S.K. Nayar,Eurographics Symposium on Rendering, Jun, 2007.Take homeThank You15-463 Class Project from
View Full Document
Unlocking...