SIGGRAPH 2000 Course on 3D Photography Passive 3D Photography Steve Seitz Carnegie Mellon University University of Washington http www cs cmu edu seitz Visual Cues Shading Merle Norman Cosmetics Los Angeles Visual Cues Shading Texture The Visual Cliff by William Vandivert 1960 Visual Cues Shading Texture Focus From The Art of Photography Canon Visual Cues Shading Texture Focus Motion Visual Cues Shading Others Texture Highlights Shadows Focus Silhouettes Inter reflections Motion Symmetry Light Polarization Shape From X X shading texture focus motion Talk Outline Overview Leading Approaches 1 Single view modeling 2 Stereo reconstruction 3 Structure from motion Single View Modeling How Do Humans Do This Good Guesswork Based on Priors these lines look parallel this looks like a cube this looks like a shadow Computers Can Do This Too Shape from shading Horn 89 User aided modeling Tour into the Picture Horry 97 Facade Debevec 96 Single View Metrology Criminisi 99 Learning approaches Morphable Models Blanz 99 Vanishing Points Vanishing Point Measuring Height 5 4 3 2 1 Same Concepts Enable Reconstructing X Y and Z Computing camera projection matrix Eliminating the ruler 5 4 3 3 2 8 Single View Metrology Criminisi 99 Single View Metrology Criminisi 99 The Music Lesson Jan Vermeer 1662 65 Royal Collection of Her Majesty Queen Elizabeth II Morphable Models Blanz 99 Video Stereo Reconstruction The Stereo Problem Shape from two or more images Biological motivation known camera viewpoints Stereo scene point image plane focal point Stereo Basic Principle Triangulation Gives reconstruction as intersection of two rays Requires point correspondence Stereo Correspondence Determine Pixel Correspondence Pairs of points that correspond to same scene point epipolar line epipolar plane epipolar line Epipolar Constraint Reduces correspondence problem to 1D search along conjugate epipolar lines Stereo Matching Algorithms Match Pixels in Conjugate Epipolar Lines Assume color of point does not change Pitfalls specularities low contrast regions occlusions image error camera calibration error Numerous approaches dynamic programming Baker 81 Ohta 85 smoothness functionals more images trinocular N ocular Okutomi 93 graph cuts Boykov 00 Structure from Motion Unknown camera viewpoints Reconstruct Scene geometry Camera motion Structure from Motion The SFM Problem Reconstruct scene geometry and camera motion from two or more images Track 2D Features Estimate 3D Optimize Fit Surfaces Structure from Motion Step 1 Track Features Detect good features corners line segments Find correspondences between frames window based correlation Structure from Motion I1 I 2 If Images 1 2 X1 X 2 X n Structure f Motion Step 2 Estimate Motion and Structure Orthographic projection e g Tomasi 92 2 or 3 views at a time Hartley 00 Structure from Motion Step 3 Refine Estimates Nonlinear optimization over cameras and points Hartley 94 Bundle adjustment in photogrammetry Structure from Motion Poor mesh Good mesh Morris and Kanade 2000 Step 4 Recover Surfaces Image based triangulation Morris 00 Baillard 99 Silhouettes Fitzgibbon 98 Stereo Pollefeys 99 Resources Computer Vision Home Page http www cs cmu edu afs cs project cil ftp html vision html Computer Vision Textbooks O Faugeras Three Dimensional Computer Vision MIT Press 1993 E Trucco and A Verri Introductory Techniques for 3 D Computer Vision Prentice Hall 1998 V S Nalwa A Guided Tour of Computer Vision Addison Wesley 1993 R Jain R Kasturi and B G Schunck Machine Vision McGraw Hill 1995 R Klette K Schluns and A Koschan Computer Vision Three Dimensional Data from Images Springer Verlag 1998 M Sonka V Hlavac and R Boyle Image Processing Analysis and Machine Vision Brooks Cole Publishing 1999 D H Ballard and C M Brown Computer Vision Prentice Hall 1982 B K P Horn Robot Vision McGraw Hill 1986 J Koenderink Solid Shape MIT Press 1990 D Marr Vision Freeman 1982 Bibliography Single View Modeling V Blanz T Vetter A Morphable Model for the Synthesis of 3D Faces SIGGRAPH 99 pp 187 194 A Criminisi I Reid A Zisserman Single View Metrology ICCV 2000 pp 434 441 B K P Horn M Brooks Shape from Shading 1989 MIT Press Cambridge M A Y Horry K Anjyo K Arai Tour into the Picture SIGGRAPH 97 pp 225 232 R Zhang P S Tsai J Cryer M Shah Shape from Shading A Survey IEEE Trans on PAMI 21 8 1999 Stereo Y Boykov O Veksler R Zabih Fast Approximate Energy Minimization via Graph Cuts ICCV 1999 Y Ohta T Kanade Stereo by Intra and Inter Scanline Search Using Dynamic Programming IEEE Trans on PAMI 7 2 1985 pp 129 154 M Okutomi T Kanade A Multiple Baseline Stereo IEEE Trans on Pattern Analysis and Machine Intelligence 15 4 1993 353 363 Bibliography Structure from Motion C Baillard A Zisserman Automatic Reconstruction of Planar Models from Multiple Views CVPR 99 pp 559 565 A W Fitzgibbon G Cross A Zisserman Automatic 3D Model Construction for TurnTable Sequences SMILE Workshop 1998 R Hartley A Zisserman Multiple View Geometry Cambridge Univ Press 2000 R Hartley Euclidean Reconstruction from Uncalibrated Views In Applications of Invariance in Computer Vision Springer Verlag 1994 pp 237 256 D Morris T Kanade Image Consistent Surface Triangulation CVPR 00 pp 332 338 M Pollefeys R Koch L Van Gool Self Calibration and Metric Reconstruction in spite of Varying and Unknown Internal Camera Parameters Int J of Computer Vision 32 1 1999 pp 7 25 C Tomasi T Kanade Shape and Motion from Image Streams Under Orthography A Factorization Method Int Journal of Computer Vision 9 2 1992 pp 137 154