Last LectureTodayPhotometric StereoSlide 4Diffuse reflectionShape from shadingPhotometric stereoSolving the equationsMore than three lightsComputing light source directionsRecall the rule for specular reflectionDepth from normalsExampleLimitationsExample-based Photometric StereoShiny thingsSlide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Virtual viewsVelvetVirtual ViewsBrushed FurSlide 28Slide 29Slide 30Slide 31Linear combinations of materialsSlide 33Slide 34Slide 35Slide 36Problem definitionFast Separation of Direct and Global Images Using High Frequency IlluminationSlide 39Slide 40Slide 41Slide 42Slide 43Slide 44Slide 45Slide 46Slide 47Slide 48Slide 49Slide 50Slide 51Slide 52Slide 53Slide 54Slide 55Slide 56Slide 57Slide 58Slide 59Slide 60Slide 61Slide 62Last Lecture•Single View ModelingVermeer’s Music LessonReconstructions by Criminisi et al.Today•Photometric Stereo•Separate Global and Direct IlluminationPhotometric StereoPhotometric StereoReadings•R. Woodham, Photometric Method for Determining Surface Orientation from Multiple Images. Optical Engineering 19(1)139-144 (1980). (PDF) Merle Norman Cosmetics, Los AngelesDiffuse reflectionSimplifying assumptions•I = Re: camera response function f is the identity function:–can always achieve this in practice by solving for f and applying f -1 to each pixel in the image•Ri = 1: light source intensity is 1–can achieve this by dividing each pixel in the image by Riimage intensity of PShape from shadingSuppose You can directly measure angle between normal and light source•Not quite enough information to compute surface shape•But can be if you add some additional info, for example–assume a few of the normals are known (e.g., along silhouette)–constraints on neighboring normals—“integrability” –smoothness•Hard to get it to work well in practice–plus, how many real objects have constant albedo?Photometric stereoNL1L2VL3Can write this as a matrix equation:Solving the equationsMore than three lightsGet better results by using more lightsWhat’s the size of LTL?Least squares solution:Solve for N, kd as beforeComputing light source directionsTrick: place a chrome sphere in the scene•the location of the highlight tells you where the light source isFor a perfect mirror, light is reflected about NRecall the rule for specular reflectionotherwise0if RVieRRWe see a highlight when V = R•then L is given as follows:Depth from normalsGet a similar equation for V2•Each normal gives us two linear constraints on z•compute z values by solving a matrix equationV1V2NExampleLimitationsBig problems•doesn’t work for shiny things, semi-translucent things•shadows, inter-reflectionsSmaller problems•camera and lights have to be distant•calibration requirements–measure light source directions, intensities–camera response functionNewer work addresses some of these issuesSome pointers for further reading:•Zickler, Belhumeur, and Kriegman, "Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction." IJCV, Vol. 49 No. 2/3, pp 215-227. •Hertzmann & Seitz, “Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs.” IEEE Trans. PAMI 2005Aaron HertzmannUniversity of TorontoExample-based Photometric StereoSteven M. SeitzUniversity of WashingtonShiny things“Orientation consistency”same surface normalVirtual viewsVelvetVirtual ViewsBrushed FurBrushed FurVirtual Views0.9+ 0.6 + 0.2 + 2.0 + 2.1 + 2.1== + += + +Linear combinations of materials0.90.60.22.02.12.1Virtual viewsphotometric stereo laser scanlaser scanphotometric stereoProblem definitionEstimate 3D shape by varying illumination, fixed cameraOperating conditions•any opaque material•distant camera, lighting•reference object available•single material (will relax later...)•no shadows, interreflections, transparencyFast Separation of Direct and Global Images Using High Frequency IlluminationShree K. NayarGurunandan G. KrishnanColumbia UniversitySIGGRAPH ConferenceBoston, July 2006Support: ONR, NSF, MERLMichael D. GrossbergCity College of New YorkRamesh RaskarMERLsourcesurfacePDirect and Global IlluminationAA : DirectBB : InterrelectionCC : SubsurfaceD participating mediumD : Volumetric translucent surfaceEE : Diffusion cameraFast Separation of Direct and Global Images• Create Novel Images of the Scene• Enhance Brightness Based Vision Methods• New Insights into Material Properties],[],[],[ icLicLicLgddirectglobalradianceDirect and Global Components: Interreflections surface icamerasourcePgjiLjiAicL ],[],[],[ jBRDF and geometryHigh Frequency Illumination Patternsurfacecamerasourcefraction of activated source elements],[],[],[ icLicLicLgd+ iHigh Frequency Illumination Patternsurfacefraction of activated source elementscamerasource],[],[],[ icLicLicLgd+-],[],[ icLicLg)1( i:21min2LLgSeparation from Two Imagesdirect global,minmaxLLLdOther Global Effects: Subsurface Scatteringtranslucent surfacecamerasource ijOther Global Effects: Volumetric Scatteringsurfacecamerasourceparticipating medium ijDiffuse InterreflectionsSpecularInterreflectionsVolumetric ScatteringSubsurfaceScattering DiffusionSceneScene Direct GlobalReal World Examples:Can You Guess the Images?Eggs: Diffuse InterreflectionsDirect GlobalWooden Blocks: Specular InterreflectionsDirect GlobalNovel ImagesMirror Ball: Failure CaseDirect GlobalPhotometric Stereo using Direct ImagesBowlShapeSource 1Source 2Source 3DirectGlobalNayar et al., 1991Kitchen Sink: Volumetric ScatteringVolumetric Scattering:Chandrasekar 50, Ishimaru 78 Direct GlobalPeppers: Subsurface ScatteringDirect GlobalHandDirect GlobalSkin: Hanrahan and Krueger 93,Uchida 96, Haro 01, Jensen et al. 01,Cula and Dana 02, Igarashi et al. 05, Weyrich et al. 05Face: Without and With MakeupGlobalDirectGlobalDirectWithout MakeupWith MakeupBlonde HairHair Scattering: Stamm et al. 77,Bustard and Smith 91, Lu et al. 00Marschner et al. 03Direct GlobalSummary• Fast and Simple Separation Method• Wide Variety of Global Effects• No Prior Knowledge of Material Properties• Implications:• Generation of Novel Images• Enhance Computer Vision Methods• Insights into Properties of