Lecture 20: Light, color, and reflectanceAnnouncementsMeasuring heightSlide 4Camera calibrationVanishing points and projection matrixCalibration using a reference objectChromaglyphsEstimating the projection matrixDirect linear calibrationSlide 11Slide 12Alternative: multi-plane calibrationSome Related TechniquesSlide 15Automatic approachesSlide 17LightSlide 19Properties of lightRadiometrySlide 22Slide 23What is light?Slide 25The light fieldCapturing light fieldsLight field exampleMore info on light fieldsSlide 30Visible lightLecture 20: Light, color, and reflectanceCS6670: Computer VisionNoah SnavelyAnnouncements•Final projects–Everyone should have received feedback by now–Midterm reports due November 24–Final presentations tentatively scheduled for the final exam period:Wed, December 16, 7:00 PM - 9:30 PMMeasuring heightRHvzrbtRHZZtvbrrvbtimage cross ratioHb0t0vvxvyMeasuring heightvzrbt0vxvyvanishing line (horizon)vt0m0What if the point on the ground plane b0 is not known?•Here the guy is standing on the box, height of box is known•Use one side of the box to help find b0 as shown aboveb0t1b1Camera calibrationGoal: estimate the camera parameters•Version 1: solve for projection matrixΠXx 1************ZYXwwywx•Version 2: solve for camera parameters separately–intrinsics (focal length, principle point, pixel size)–extrinsics (rotation angles, translation)–radial distortionVanishing points and projection matrix************Π 4321ππππ1π2π3π4π T00011Ππ = vx (X vanishing point)Z3Y2 , similarly,vπvπ origin worldof projection10004TΠπ ovvvΠZYXNot So Fast! We only know v’s up to a scale factor ovvvΠZYXcba•Can fully specify by providing 3 reference pointsCalibration using a reference objectPlace a known object in the scene•identify correspondence between image and scene•compute mapping from scene to imageIssues•must know geometry very accurately•must know 3D->2D correspondenceChromaglyphsCourtesy of Bruce Culbertson, HP Labshttp://www.hpl.hp.com/personal/Bruce_Culbertson/ibr98/chromagl.htmEstimating the projection matrixPlace a known object in the scene•identify correspondence between image and scene•compute mapping from scene to imageDirect linear calibrationDirect linear calibrationCan solve for mij by linear least squares•use eigenvector trick that we used for homographiesDirect linear calibrationAdvantage:•Very simple to formulate and solveDisadvantages:•Doesn’t tell you the camera parameters•Doesn’t model radial distortion•Hard to impose constraints (e.g., known focal length)•Doesn’t minimize the right error functionFor these reasons, nonlinear methods are preferred•Define error function E between projected 3D points and image positions–E is nonlinear function of intrinsics, extrinsics, radial distortion•Minimize E using nonlinear optimization techniques–e.g., variants of Newton’s method (e.g., Levenberg Marquart)Alternative: multi-plane calibration Images courtesy Jean-Yves Bouguet, Intel Corp.Advantage•Only requires a plane•Don’t have to know positions/orientations•Good code available online!–Intel’s OpenCV library: http://www.intel.com/research/mrl/research/opencv/ –Matlab version by Jean-Yves Bouget: http://www.vision.caltech.edu/bouguetj/calib_doc/index.html–Zhengyou Zhang’s web site: http://research.microsoft.com/~zhang/Calib/Some Related TechniquesTour Into The PictureAnjyo et al., SIGGRAPH 1997http://koigakubo.hitachi.co.jp/little/DL_TipE.htmlSome Related TechniquesImage-Based Modeling and Photo Editing•Mok et al., SIGGRAPH 2001•http://graphics.csail.mit.edu/ibedit/ Single View Modeling of Free-Form Scenes•Zhang et al., CVPR 2001•http://grail.cs.washington.edu/projects/svm/Automatic approachesMake3D, Ashutosh SaxenaAutomatic approachesD. Hoiem, A.A. Efros, and M. Hebert, "Automatic Photo Pop-up", ACM SIGGRAPH 2005.Lightby Ted AdelsonReadings•Szeliski, 2.2, 2.3.2Lightby Ted AdelsonReadings•Szeliski, 2.2, 2.3.2Properties of lightToday•What is light?•How do we measure it?•How does light propagate?•How does light interact with matter?RadiometryWhat determines the brightness of an image pixel?Light source propertiesLight source propertiesSurface propertiesSurface propertiesRadiometryWhat determines the brightness of an image pixel?RadiometryWhat determines the brightness of an image pixel?Light sourcepropertiesSurface shapeSurface reflectancepropertiesOpticsSensor characteristicsSlide by L. Fei-FeiExposureWhat is light?Electromagnetic radiation (EMR) moving along rays in space•R() is EMR, measured in units of power (watts)–  is wavelengthLight field•We can describe all of the light in the scene by specifying the radiation (or “radiance” along all light rays) arriving at every point in space and from every directionThe light field•Known as the plenoptic function•If you know R, you can predict how the scene would appear from any viewpoint.The light field•Assume radiance does not change along a ray–what does this assume about the world?•Parameterize rays by intersection with two planes:•Usually drop  and time parameters•How could you capture a light field?t is not time (different from above t !)Capturing light fieldsStanford/Cornell spherical gantryStanford Multi-Camera ArrayLego Mindstorms GantryHandheld light field cameraLight field exampleMore info on light fieldsIf you’re interested to read more:The plenoptic function•Original reference: E. Adelson and J. Bergen, "The Plenoptic Function and the Elements of Early Vision," in M. Landy and J. A. Movshon, (eds) Computational Models of Visual Processing, MIT Press 1991.•L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System”, Proc. SIGGRAPH, 1995, pp. 39-46.The light field•M. Levoy and P. Hanrahan, “Light Field Rendering”, Proc SIGGRAPH 96, pp. 31-42.•S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proc. SIGGRAPH, 1996, pp. 43-54.What is light?Electromagnetic radiation (EMR) moving along rays in space•R() is EMR, measured in units of power (watts)–  is wavelengthPerceiving light•How do we convert radiation into “color”?•What part of the spectrum do we see?Visible lightWe “see” electromagnetic