Camera calibration & radiometryCamera calibrationCalibration targetFrom last lecture: camera calibrationCamera calibrationCamera calibrationWhat makes a valid M matrix?Camera calibrationIrradiance, ERadiance, LWhat you’d like to pull out from LSpecial case BRDF: Lambertian reflectanceReflectance mapRelate surface normal to p & qRefl map for point source (in direction ) Lambertian surfacePicture of Lambertian refl mapWhat constraints are there for form of reflectance map?Let’s list the things this model doesn’t handle properlyLinear shading mapLinear shading: 1st order terms of Lambertian shadingLinear shadingAdvantages of linear shadingKnowing the reflectance map, can we infer the gradient at any point?Generic reflectance mapPhotometric stereoApproach 1Approach 2Photometric stereoFrom the image under the ith lighting condition (Lambertian)Combining all the measurementsSolve for g(x,y). May be ill-conditionedA fix to avoid problems in dark areas: pre-multiply both sides by the image intensitiesRecovering albedo and surface normalSurface shape from surface gradientsCamera calibration & radiometry• Reading: – Chapter 2, and section 5.4, Forsyth & Ponce– Chapter 10, Horn• Optional reading:– Chapter 4, Forsyth & PonceSept. 17, 2002MIT 6.801/6.866Profs. Freeman and DarrellCamera calibration• Geometric: how positions in the image relate to 3-d positions in the world.• Photometric: how the intensities in the image relate surface and lighting properties in the world.Calibration targethttp://www.kinetic.bc.ca/CompVision/opti-CAL.htmlFrom last lecture: camera calibration=1.........11321zyxTTTWWWmmmzvuPMzprr1=pixel coordinatesworld coordinatesz is in the camera coordinate system, but we can solve for that, since , leading to:zPmr⋅=31PmPmvPmPmurrrr⋅⋅=⋅⋅=3231Camera calibrationPmPmvPmPmurrrr⋅⋅=⋅⋅=3231Because of these relations,0)(0)(3231=⋅−=⋅−iiiiPmvmPmumrrFor each feature point, i, we have:=−−−−−−−−−−−−−−−−00001000000001100000000134333231242322211413121111111111111111111111MLLLmmmmmmmmmmmmvPvPvPvPPPuPuPuPuPPPvPvPvPvPPPuPuPuPuPPPnnznnynnxnnznynxnnznnynnxnnznynxzyxzyxzyxzyxP m = 0Camera calibrationWe want to solve for the unit vector m (the stacked one)that minimizes2PmThe eigenvector corresponding to the minimum eigenvalue of the matrix PTP gives us that (see Forsyth&Ponce, 3.1).What makes a valid M matrix?defined only up to a scale;normalize M so that.133==TTrarr()== baaaMTTTrrrrr321bA.M is a perspective projection matrix iff 0)(≠ADet.Camera calibration• Geometric: how positions in the image relate to 3-d positions in the world.• Photometric: how the intensities in the image relate surface and lighting properties in the world.surfacelightIrradiance, E• Light power per unit area (watts per square meter) incident on a surface.surfacelightRadiance, L• Amount of light radiated from a surface into a given solid angle per unit area (watts per square meter per steradian).• Note: the area is the foreshortened area, as seen from the direction that the light is being emitted.Horn, 1986),(),(),,,(iieeeeiiELfBRDFφθφθφθφθ==How does the world give us the brightness we observe at a point?Accounting for the foreshortened area of center patch relative to illuminant.The total irradiance of the surface is:iiiiiddEEφθθθφθπππ )cos( )sin(),(i2/00∫∫−=radiance per solid angleThe total radiance reflected from the surface patch is:iiiiieeiieeddEfLφθθθφθφθφθφθπππ )cos( )sin( ),( ),,,(),(i2/0∫∫−=What you’d like to pull out from LPixel intensities may be proportional to radiance reflected from the surface patch:iiiiieeiieeddEfLφθθθφθφθφθφθπππ )cos( )sin( ),( ),,,(),(i2/0∫∫−=eeφθ,surface orientation relative to camera),,,(eeiifφθφθsurface BRDF),(iiEφθillumination conditionsiiφθ,surface orientation relative to illuminationThat’s hard, so let’s focus on special cases for the rest of this lecture.Special case BRDF: Lambertianreflectanceπφθφθ1),,,( =eeiifiiiiieeddLφθθθφφδθθδπφθπππ )cos( )sin( )()( 1),(i002/0−−=∫∫−Radiance reflected from Lambertian surface illuminated by point source:BRDF is a constant. These surfaces look equally bright from all viewing directions.)cos(0θ∝Reflectance map• For orthographic projection, and light sources at infinity, the reflectance map is a useful tool for describing the relationship of surface orientation to image intensity.• Describes the image intensity for a given surface orientation.• Parameterize surface orientation by the partial derivatives p and q of surface height z.Relate surface normal to p & q11pxz=∂∂qyz=∂∂xykzqypx =−+Local tangent plane:nˆUnit normal to surface:()2211ˆqpqpnnnT++−−==rrRefl map for point source (in direction ) Lambertian surfacesˆFor a Lambertian surface,)cos(ˆˆ),(isnqpRθ=⋅∝2222111ssssqpqpqqppk++++++=Unit vector to source:()2211ˆssTssqpqps++−−=Picture of Lambertian refl mapHorn, 1986W. T. Freeman, Exploiting the generic viewpoint assumption, International Journal Computer Vision, 20 (3), 243-261, 1996What constraints are there for form of reflectance map?Freeman, 1994Freeman, 1994Freeman, 1994Shape 1Freeman, 1994Shape 2Freeman, 1994shape 1 shape 2Freeman, 1994Freeman, 1994Freeman, 1994Freeman, 1994Freeman, 1994Let’s list the things this model doesn’t handle properly• Occluding edges• Albedo changes• Perspective effects (small)• Interreflections• Material changes across surfaces in the imageLinear shading mapHorn, 1986Linear shading: 1storder terms of Lambertian shading2222111),(ssssqpqpqqppkqpR++++++=Lambertian point sourceqqqpRppqpRkqpqp 0,00,02),(),( ====∂∂+∂∂+≈1storder Taylor series about p=q=0)1( 2qqppkss++=See Pentland, IJCV vol. 1 no. 4, 1990.Linear shadingrange imageLambertian shadingquadratic terms higher-order termslinear shadingPentland 1990, Adelson&Freeman, 1991Advantages of linear shading• Linear relationship between surface range map and rendered image.• Rendering is easy: