Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 16: Shape from XInitialize‡Spell check offIn[1]:=Off@General::spell1D;In[2]:=SetOptions@ArrayPlot, ColorFunction Ø "GrayTones", DataReversed Ø True,Frame Ø False, AspectRatio Ø Automatic, Mesh Ø False,PixelConstrained Ø True, ImageSize Ø SmallD;SetOptions@ListPlot, ImageSize Ø SmallD;SetOptions@Plot, ImageSize Ø SmallD;SetOptions@DensityPlot, ImageSize Ø Small, ColorFunction Ø GrayLevelD;nbinfo = NotebookInformation@EvaluationNotebook@DD;dir =H"FileName" ê. nbinfo ê. FrontEnd`FileName@d_List, nam_, ___D ßToFileName@dDL;OutlineLast time‡Geometry,shape and depth: Representation issues & generative models‡Lambertian model & surface normal representationsSurface normal representation: Local, dense, metric, viewer-dependent coordinate system (e.g. slant from observer).Today‡Inference of shape from shading--set in context of other cues to shape, ‡Perception of shape from shading‡Formal ambiguities in the generative model, the bas-relief transform‡Task analysis: "Shape for X" problemPerception of "Shape from X" What is shape? One mathematical definition: "Geometrical properties that are invariant over translation and scale". A computational vision definition: "Whatever is left over after discounting material, illumination and viewpoint variations in the images of an object"We will take a more general view here:"geometrical relationships within a surface that are useful for visual functions". More precise definitions depend on the function.Later we will return to the "discounting" issues.There are a variety of cues to shape. In natural images, these cues typically co-vary. Human vision can infer shape from any of them or in combination, hence "shape from X". Later on we will talk about cue integration. In a psychophysics lab, they can be manipulated independently. Let's look at several categories of shape cues.TextureLater on we will talk in more detail about how to model and infer surface material properties. Texture is one property of surfaces that is particularly informative for shape and object identity. Texture by definition has some degree of regularity. Texture can be highly regular, or only regular in the statistical sense. For the moment, suppose a surface has a determinis-tic regular homogeneous pattern of "texture elements", where each texture element is the same 3D size, and they are distributed homogeneously.2 16.ShapeFromX.nb‡Slant & textureDefine a function to place circular disks on a regular gridIn[11]:=checker[x_,y_,space_,radius_]:= If[(Mod[x,space]^2+Mod[y,space]^2<radius^2) || (Mod[-x,space]^2+Mod[-y,space]^2<radius^2) || (Mod[x,space]^2+Mod[-y,space]^2<radius^2) || (Mod[-x,space]^2+Mod[y,space]^2<radius^2),0,1];pic = Table[{checker[x, y, 32, 12]}, {x, 1, 512}, {y, 1, 512}];ListPlot3D@Table@1, 8x, -5, 5, .1<, 8y, -5, 5, .1<D, Mesh Ø None,VertexColors Ø 8pic@@5 ;; -5 ;; 5, 5 ;; -5 ;; 5DD<, Lighting Ø "Neutral",Axes -> False, Boxed -> False, ImageSize Ø SmallDOut[24]=In[19]:=pic = Reverse@ExampleData@8"TestImage", "Mandrill"<, "Data"D ê 255.D;Try changing the orientation of the above textureThe generative assumptions above lead to regularities in the image that can be used to estimate surface slant--if the spatial scale of the texture is small in comparison to the surface curvature, so that the surface can be approximated locally by a plane. Then, cues to surface slant: 1) spatial gradient of the density of texture elements: the image density increases with distance; 2) an individual element (such as a circular disk) gets smaller with distance, and its image height to width ratio gets smaller ("compressed") with increasing slant;3) the ratio of the back width to the front width of an individual element gets smaller with slant (imagine a small square, linear pespective on small scale) . (See Knill). Tilt is the direction that the surface slants away most rapidly. Here the density and sizes of the elements change the most rapidly.Shape and texture: Adapt the above function to map a texture onto a smooth shape16.ShapeFromX.nb 3Shape and texture: Adapt the above function to map a texture onto a smooth shapeProject idea: Map a small single textural element onto an "invisible" bump. Allow the user to move the element around. Can you infer the underyling shape?Stereo disparity‡Random dot stereogramsA simple planar surface in depth.http://demonstrations.wolfram.com/FloatingDiskStereogram/Cross your eyes so that you see three dots (rather than two) at the bottom. The above surface should appear like the one below, but viewed from above.But smooth shape from stereo can also be conveyed through disparity.4 16.ShapeFromX.nb‡Random dot stereograms, "Magic eye" style In[110]:=H*Copyright HcL Dror Bar-Natan 1994*LH*Random Dot Stereograms in The Mathematica Journal 1-3 H1991L 69-75L.*Lf@x_, y_D := Sin@20 Hx^2 + y^2LD ê H1 + 5 Hx^2 + y^2LLH*the function drawn*LShow@Graphics@Table@H*loop 500 times*Lx = Random@Real, 8-1, -0.6<D;H*generate a random number between-1 and-0.6*Ly = Random@Real, 8-1, 1<D;H*generate a random number between-1 and 1*LTable@H*loop 6 times*Lx0 = x; x = x + 0.4 + f@x, yD ê 15;H*compute coordinates of next pixel*LPoint@8x0, y<D ,H*put a point*L86<D, H*end inner loop*L8500<DDDH*end outer loop*LOut[111]=Here's one at a higher dot density:16.ShapeFromX.nb 5Here's one at a higher dot density:See: Christopher Tyler. The wiki page provides nice summary: http://en.wikipedia.org/wiki/Autostereogram.‡Stereo + shading + surface contoursIn[61]:=pL=Plot3D[Cos[Sqrt[x^2+y^2]], {x,-10,10}, {y, -10, 10},AspectRatio -> Automatic, PlotPoints-> {25,25}, ViewPointØ{1.3,-2.4,2.}];6 16.ShapeFromX.nbIn[62]:=Manipulate@pR = Plot3D@Cos@Sqrt@x^2 + y^2DD, 8x, -10, 10<, 8y, -10, 10<,AspectRatio Ø Automatic, PlotPoints Ø 825, 25<,ViewPoint Ø 8vp, -2.4, 2.<D;Show@GraphicsArray@8pL, pR<DD, 88vp, 1.1<, 0.8, 1.5<DOut[62]=vpMotionWe'll look at structure & shape from motion in detail later. With some care, one can put up an animation in which any single frame shows no apparent structure, but when moving, the shape becomes clear. E.g. rotating "glass" cylinder with dots on it. Or, "biological motion".random.movcube.mov16.ShapeFromX.nb 7Contours‡Surface contour (markings)In[63]:=Plot[{Sin[x],Sin[x]+1,Sin[x]+2, Sin[x]+3, Sin[x]+4, Sin[x]+5},{x,0,10}, Axes->False,ImageSize->Small]Out[63]=‡Shape from moving contoursHere's an example taken from the Mathematica documentation. Apparent 3D shape