Scene Modeling for a Single ViewBreaking out of 2Don to 3D…Camera rotations with homographiesCamera translationYes, with planar scene (or far away)So, what can we do here?Some preliminaries: projective geometrySilly Euclid: Trix are for kids!The projective planeProjective linesPoint and line dualityIdeal points and linesVanishing pointsVanishing points (2D)Slide 16Vanishing linesSlide 18Computing vanishing pointsComputing vanishing linesSlide 21Fun with vanishing points“Tour into the Picture” (SIGGRAPH ’97)The ideaFitting the box volumeSlide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 352D to 3D conversionSlide 37Depth of the boxDEMOForeground ObjectsSlide 41Foreground DEMO (and video)Scene Modeling for a Single View15-463: Computational PhotographyAlexei Efros, CMU, Fall 2006René MAGRITTEPortrait d'Edward James …with a lot of slides stolen from Steve Seitz and David Brogan,Breaking out of 2D…now we are ready to break out of 2DAnd enter the real world!Enough of images!We want more of the plenoptic functionWe want real 3D scenewalk-throughs:Camera rotationCamera translationCan we do it from a single photograph?on to 3D…Camera rotations with homographiesSt.Petersburgphoto by A. TikhonovVirtual camera rotationsOriginal imageCamera translationDoes it work?synthetic PPPP1PP2Yes, with planar scene (or far away)PP3 is a projection plane of both centers of projection, so we are OK!PP1PP3PP2So, what can we do here?Model the scene as a set of planes!Now, just need to find the orientations of these planes.Some preliminaries: projective geometryAmes RoomSilly Euclid: Trix are for kids!Parallel lines???(0,0,0)The projective planeWhy do we need homogeneous coordinates?•represent points at infinity, homographies, perspective projection, multi-view relationshipsWhat is the geometric intuition?•a point in the image is a ray in projective space(sx,sy,s)•Each point (x,y) on the plane is represented by a ray (sx,sy,s)–all points on the ray are equivalent: (x, y, 1) (sx, sy, s)image plane(x,y,1)yxzProjective linesWhat does a line in the image correspond to in projective space?•A line is a plane of rays through origin–all rays (x,y,z) satisfying: ax + by + cz = 0 zyxcba0 :notationvectorin•A line is also represented as a homogeneous 3-vector ll plPoint and line duality•A line l is a homogeneous 3-vector•It is to every point (ray) p on the line: l p=0p1p2What is the intersection of two lines l1 and l2 ?•p is to l1 and l2 p = l1 l2Points and lines are dual in projective space•given any formula, can switch the meanings of points and lines to get another formulal1l2pWhat is the line l spanned by rays p1 and p2 ?•l is to p1 and p2 l = p1 p2 •l is the plane normalIdeal points and linesIdeal point (“point at infinity”)•p (x, y, 0) – parallel to image plane•It has infinite image coordinates(sx,sy,0)yxzimage planeIdeal line•l (0, 0, 1) – parallel to image planeVanishing pointsVanishing point•projection of a point at infinity•Caused by ideal lineimage planecameracenterground planevanishing pointVanishing points (2D)image planecameracenterline on ground planevanishing pointVanishing pointsProperties•Any two parallel lines have the same vanishing point v•The ray from C through v is parallel to the lines•An image may have more than one vanishing pointimage planecameracenterCline on ground planevanishing point Vline on ground planeVanishing linesMultiple Vanishing Points•Any set of parallel lines on the plane define a vanishing point•The union of all of these vanishing points is the horizon line–also called vanishing line•Note that different planes define different vanishing linesv1v2Vanishing linesMultiple Vanishing Points•Any set of parallel lines on the plane define a vanishing point•The union of all of these vanishing points is the horizon line–also called vanishing line•Note that different planes define different vanishing linesComputing vanishing pointsProperties•P is a point at infinity, v is its projection•They depend only on line direction•Parallel lines P0 + tD, P1 + tD intersect at PVDPP t00/1///1ZYXZZYYXXZZYYXXtDDDttDtPDtPDtPtDPtDPtDPPPΠPvP0DComputing vanishing linesProperties•l is intersection of horizontal plane through C with image plane•Compute l from two sets of parallel lines on ground plane•All points at same height as C project to l–points higher than C project above l•Provides way of comparing height of objects in the sceneground planelCFun with vanishing points“Tour into the Picture” (SIGGRAPH ’97)Create a 3D “theatre stage” of five billboardsSpecify foreground objects through bounding polygonsUse camera transformations to navigate through the sceneThe ideaMany scenes (especially paintings), can be represented as an axis-aligned box volume (i.e. a stage)Key assumptions:•All walls of volume are orthogonal•Camera view plane is parallel to back of volume•Camera up is normal to volume bottomHow many vanishing points does the box have?•Three, but two at infinity•Single-point perspectiveCan use the vanishing pointto fit the box to the particularScene!Fitting the box volumeUser controls the inner box and the vanishing point placement (# of DOF???)Q: What’s the significance of the vanishing point location?A: It’s at eye level: ray from COP to VP is perpendicular to image plane. Why?High CameraExample of user input: vanishing point and back face of view volume are definedHigh CameraExample of user input: vanishing point and back face of view volume are definedLow CameraExample of user input: vanishing point and back face of view volume are definedLow CameraExample of user input: vanishing point and back face of view volume are definedHigh Camera Low CameraComparison of how image is subdivided based on two different camera positions. You should see how moving the vanishing point corresponds to moving the eyepoint in the 3D world.Left CameraAnother example of user input: vanishing point and back face of view volume are definedLeft CameraAnother example of user input: vanishing point and back face of view volume are definedRightCameraAnother example of user input: vanishing point and back face of view volume are
View Full Document
Unlocking...