1Course ReviewCS/ECE 181bSpring 2004Topics since Midterm• Stereo vision• Shape from shading• Optical flow• Face recognition projectMultiview Geometry and Stereo VisionReading: sldeis, handout#6, and Chapter 8 fromH&Z2Questions• Correspondence geometry: Given an image point x in the firstview, how does this constrain the position of the correspondingpoint x’ in the second image?• Camera geometry (motion): Given a set of correspondingimage points {xi _x’i}, i=1,…,n, what are the cameras P and P’ forthe two views?• Scene geometry (structure): Given corresponding imagepoints xi _x’i and cameras P, P’, what is the position of (theirpre-image) X in space?M. PollefeysEpipolar geometry• Epipolar Plane• Epipoles• Epipolar Lines• BaselineC1C2Essential MatrixOOPOPO POO[]RttttttRtExyxzyz ==000E - Essential Matrix3Case 2: Uncalibrated camera• Intrinsic parameters not known0ˆˆ=pEpT00)()(0)()(1211211===pFppKEKppKEpKTTTTpKpˆ1=pKp=ˆ2121= KEKFTFundamental Matrix=100sin0cot00vuK Points in the normalized image planegeometric derivationxHx'=x'e'l' =[]FxxHe'==mapping from 2-D to 1-D family (rank 2)Fundamental Matrix FThe Fundamental Matrix• F has seven independent parameters• A simple, linear technique to recover F fromcorresponding point locations is the “eight pointalgorithm”• From F, we can recover the epipolar geometry of thecameras– Not saying how…• This is called weak calibration4StereopsisBasic Stereo Configuration: rectified imagesxrxlZXbfXb+2Xb2xfXbZl=+2xfXbZr=2xxfbZZbfx xlrlr==()ZZ2bDisparityStereo disparity• “Stereo disparity” is the difference in position betweencorrespondence points in two images– Disparity is inversely proportional to scene depth(u0, v0)(u0, v0)Disparity: (du0, dv0) = (u0 - u0, v0 - v0) = (0, 0)Disparity is a vector!5Random Dot StereogramsHow is this possible with completely random correspondence?Left Right Depth imageStereo: Summary• Multiview geometry– Epipolar geometry• Correspondence problem• Essential Matrix and Fundamental Matrix• Stereopsis: stereo matching, disparity and depth• Random dot stereogramsShape from shadingReading: handout #7 and slides6Shape from shading• Radiance and Irradiance• Lambertian and Specular surfaces• Bidirectional reflectance distribution function (BRDF)• Fundamental Radiometric Relation• Gradient Space• Reflectance Map• Photometric StereoThree surface reflectance functions/modelsIdeal diffuse(Lambertian)DirectionaldiffuseIdealspecular7Bidirectional Reflectance Distribution Function• The BRDF tells us how bright a surface appears whenviewed from one direction while light falls on it fromanother one– General model of local reflection• More precisely, it is the ratio of reflected radiance dLr inthe direction toward the viewer to the irradiance dEi in thedirection toward the light source=  0),,,(:ooiifBRDFReflected energyIncident energyioNNBRDF modelsFor many surfaces, a simple BRDF suffices• Lambertian (diffuse, matte) surface (e.g., white powder)– Independent of exit angleLooiif  =),,,( ===otherwise 0 and if sincos1),,,(oioiooiif• Specular surface (e.g., a mirror)• Combinations (Phong, Lambertian+Specular, …)Gradient Space Representation• Orientation of a vector in 3-D space has two degrees offreedom.• Suppose we are interested in representing all vectors in aparticular hemisphere, sayz < 0 hemisphere:– We can then represent any such vector with a negative zcomponent as (p, q). See next slide.8Gradient spaceGradient SpaceLet the imaged surface be Then its surface normal can be obtained as a crossproduct of the two surface vectors:Surface normal: Reflectance Map• Reflectance map captures the dependence of brightness on surface orientation.• At a particular point in the image, we measure the image irradiance E(x,y).• This irradiance is proportional to the radiance at the corresponding point on the surfaceimaged.• If the surface gradient at that point is (p,q ), then the radiance there is R(p,q ).– This assumes or ignores other contributing factors such as reflectance properties ofthe surface or distribution of light sources• Normalizing the proportionality constant, we get:E(x,y) = R(p,q)Image irradiance equation9Lambertian surfaceLambertian surface: appears equally bright from allviewing angles.Let the incident light direction beReflectance Map• Illuminant direction: - [1 0.5 -1]• Isobrightness contours of a reflectance map of a Lambertian surfaceare a set of conic sections in gradient space.Photometric Stereo• Two images, taken with different lighting, will yield twoequations for each image point.• If these equations are linear and independent, there will bea unique solution for p and q.• For best results, the two light source directions should befar apart in gradient space.• For Lambertian surfaces, these lead to non-linear equations;there can be two solutions, one solution, or none,depending on the particular values of the intensity.10Shape from shading• Radiance and Irradiance• Lambertian and Specular surfaces• Bidirectional reflectance distribution function (BRDF)• Fundamental Radiometric Relation• Gradient Space• Reflectance Map• Photometric StereoMotion field and optical flowReading: Handout #8 and slidesMF  OFConsider a smooth, lambertian, uniform sphere rotatingaround a diameter, in front of a camera:–MF  0 since the points on the sphere are moving–OF = 0 since there are no moving patterns in the images3D3DImageImageOctavia Camps11Brightness Constancy Equation• Let P be a moving point in 3D:– At time t, P has coords (X(t),Y(t),Z(t))– Let p=(x(t),y(t)) be the coords. of its image at time t.– Let I(x(t),y(t),t) be the brightness at p at time t.• Brightness Constancy Assumption:– As P moves over time, I(x(t),y(t),t) remains constant.Octavia Campsno spatial change in brightness induce no temporalchange in brightness no discernible motionmotion perpendicular to local gradient induce notemporal change in brightness no discernible motionmotion in the direction of local gradient inducetemporal change in brightness discernible motiononly the motion component in the direction of localgradient induce temporal change