Introduction to Neural NetworksDaniel KerstenComputing object properties: Surface MaterialInitialize‡Spell check offOff@General::spell1D;SetOptions@ArrayPlot, ColorFunction Ø "GrayTones", DataReversed Ø True,Frame Ø False, AspectRatio Ø Automatic, Mesh Ø False,PixelConstrained Ø True, ImageSize Ø SmallD;SetOptions@ListPlot, ImageSize Ø SmallD;SetOptions@Plot, ImageSize Ø SmallD;SetOptions@DensityPlot, ImageSize Ø Small, ColorFunction Ø GrayLevelD;nbinfo = NotebookInformation@EvaluationNotebook@DD;dir =H"FileName" ê. nbinfo ê. FrontEnd`FileName@d_List, nam_, ___D ßToFileName@dDL;Introduction to material perception: How does the brain enable us to see what things are made of? A basic assumption about the function of the ventral visual pathway of the brain is that the neural circuits are computing properties of objects that are invariant to changes in viewpoint and lighting. Another way of expressing the problem is to say that the visual system is discounting (or “integrating out” in Bayesian terms) unwanted variations in viewpoint and lighting. One aspect of this process is to compute representations of the form or shape of objects. Another is computing the intrinsic surface properties--the stuff, substance or material that an object is made of (cf. Cant et al., 2008). We first start by understanding what we know about the physical generative process that produces the patterns of light as a function of material properties. Material & Texture modelingGeneral categories of the "stuff" we see: surfaces (opaque and transparent), particle clouds (e.g. smoke, mist,..), liquids, hair, fur,...Connection with count vs. mass nounsGeneral categories of the "stuff" we see: surfaces (opaque and transparent), particle clouds (e.g. smoke, mist,..), liquids, hair, fur,...Connection with count vs. mass nounsUniform materialsSurfaces with material properties or attributes:reflectance ("paint" or pigment or albedo)matte and shinymirrorstransparencymultiplicative, additive‡Physics-based generative modeling: Bidirectional reflectance distribution functionsFigure from: Image-Based BRDF Measurement Including Human Skin Stephen R. Marschner* Stephen H. Westin Eric P. F. Lafortune, Kenneth E. Torrance Donald P. GreenbergThe Bidirectional Reflectance Distribution Function (BRDF) describes directional dependence of the reflected light energy. The BRDF represents, for each incoming angle, the amount of light that is scattered in each outgoing angle. For a given wavelength, it is the ratio of the reflected radiance in a particular direction to the incident irradiance:r(qi,fi,qe,fe)=dLeH,qe,feLdEiHqi,fiLwhere E is the irradiance, that is the incident flux per unit area (w-m-2), and L is the reflected radiance, or the reflected flux per unit area per unit solid angle (w-m-2-sr-1). The units of BRDF are inverse steradians. Respects the physics: Reciprocity, energy conservation.We've assumed isotropy, i.e. the BRDF is the same for all directions at a point, and spatially uniform material. For a Lambertian (perfectly diffuse) surface, for example, the BRDF is constant. The Phong model described earlier in the context of shape-from-shading can approximate only a subset of surfaces characterized by BRDFs.2 23.SurfaceMaterial.nbwhere E is the irradiance, that is the incident flux per unit area (w-m-2), and L is the reflected radiance, or the reflected flux per unit area per unit solid angle (w-m-2-sr-1). The units of BRDF are inverse steradians. Respects the physics: Reciprocity, energy conservation.We've assumed isotropy, i.e. the BRDF is the same for all directions at a point, and spatially uniform material. For a Lambertian (perfectly diffuse) surface, for example, the BRDF is constant. The Phong model described earlier in the context of shape-from-shading can approximate only a subset of surfaces characterized by BRDFs.Figure from: http://graphics.stanford.EDU/~smr/brdf/bv/‡Ward reflection model: For calculating an image from a description of the shape, the illumination, and the BRDFThe Ward model is a physically realizable cousin of the Phong model.Subscripts i and e below indicate incoming and outgoing rays, respectively.Le(qe,fe)=ŸŸLi(qi,fi)r(qi,fi,qe,fe)cosqisinfidqidfir(qi,fi,qe,fe)=rdp+rse-tan2HdLëa24 pa2cosqicosqed is the angle between the viewer and the vector defining the mirror reflection of the incident ray (i.e. where the angle of reflection equals the angle of incidence). a can be thought of is a measure of "roughness", and rd and rs give the amounts of diffuse and reflected contributions.23.SurfaceMaterial.nb 3d is the angle between the viewer and the vector defining the mirror reflection of the incident ray (i.e. where the angle of reflection equals the angle of incidence). a can be thought of is a measure of "roughness", and rd and rs give the amounts of diffuse and reflected contributions.Non-uniform materials: TextureNote: "texture" sometimes refers to low-level cues or statistics useful for inferring properties like slant and shape, but it is also used to refer to surface material properties that are useful to estimate because they represent view-invariant object properties. In other words, sometimes it refers to a cue (measurement to support an estimate) and other times to an esti-mate itself. Thus confusingly, sometimes "texture" refers to an image features, and other times to 3D surface properties.In this lecture, we focus on texture as a material property.Textures can:be regular ("herringbone pattern") or stochastic ("fur")"cohere" e.g. asphalt vs. sand & gravelTextures can be due to:reflectance/pigment variations or bump (small geometric) variationsperceptually it isn't always easy to tell the difference, and may not matter depending on visual function.For example, consider the visual and tactile differences between real wood and synthetic wood finishes.(Note that image texture can also result from a completely uniform (shiny) material reflecting a textured environment)Key characteristics of texture: spatial variations are small with respect to the global scale of the surface structurespatial homogeneity4 23.SurfaceMaterial.nb‡Appearance-based measurementsHow can one characterize the generative model? Much more complicated because of small, but not micro-scale surface non-uniformity.http://www1.cs.columbia.edu/CAVE/projects/btf/‡Random synthesis and learning of texturesThere has been considerable work in computer graphics, computer vision, and