Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 7: Image Modeling, Linear Systems‡Initialize standard library files:In[1]:=Off@General::spell1D;OutlineLast time‡Bayesian decision theory applied to perceptionThe task is important: Some causes of image features are important to estimate (primary variables, Sprim), and some are not Ssec.Use the joint probability to characterize the "universe" of possibilities:(1)p ISprim, Ssec, IM(Directed) Graphs provide a way of modeling what random variables influences other random variables, and correspond-ingly how to decompose the joint into a simpler set of factors (conditional probabilities).Through conditioning on what is known, and integrating out what we don't care to estimate, we arrive at the posterior:(2)p ISprim»IMAnd by Bayes' theorem the posterior probability is given by:p@Sprim»ID =p@I »SprimDp@SprimDp@IDp@I »SprimDis the likelihood, and is determined by the generative model for Ip@SprimDis the prior model,which can also be considered part of the generative modelp@IDis fixedThe task determines how to use the posterior.Picking the most probable value of Sprim (MAP) results in the fewest errors on average‡Counter-intuitive consequences:‡Counter-intuitive consequences:Inference: Fruit classification example Pick most probable color--Answer "red"Pick most probable fruit--Answer "apple"Pick most probable fruit AND color--Answer "red tomato", not "red apple"Moral of the story: Optimal inference depends on the precise definition of the task‡Generalization to degrees of desired precision leads to Bayesian decision theorySlant estimation example illustrates how utility affects optimal estimatesProjection ambiguity and the estimation of pose (slant) and object dimensions (aspect ratio).In[7]:=GraphicsRow@8Graphics3D@8EdgeForm@D, Cylinder@880, 0, 0<, 8.01, .01, .001<<, 1D<,ImageSize Ø Tiny, Boxed Ø False, AspectRatio Ø 1D,Graphics3D@8EdgeForm@D, Cylinder@880, 0, 0<, 8.01, .01, .001<<, 1D<,ImageSize Ø Tiny, Boxed Ø False, AspectRatio Ø 1 ê2D<DOut[7]=Imagine the image of an ellipse arrives at your retina, and for simplicity that somehow you know the width = 1. Your visual system measures the height of the ellipse in the image (x=1/2) and using this single number must estimate two object properties: 1) the aspect ratio of the actual disk causing the elliptical image, (i.e. the physical dimension of the disk orthogonal to the width); 2) the angle of inclination (slant) away from the vertical. 2 7.ImageModelLinearSystems.nbImagine the image of an ellipse arrives at your retina, and for simplicity that somehow you know the width = 1. Your visual system measures the height of the ellipse in the image (x=1/2) and using this single number must estimate two object properties: 1) the aspect ratio of the actual disk causing the elliptical image, (i.e. the physical dimension of the disk orthogonal to the width); 2) the angle of inclination (slant) away from the vertical. 7.ImageModelLinearSystems.nb 3TodayGenerative modeling -- How can we characterize an image, or a collection of images?Introduction to modeling image structure: image-based vs. scene-basedLinear systems (linear image-intensity based modeling)Optical Resolution: Diffraction limits to spatial resolutionPoint spread function, convolutions and blurOverview of image modelingGenerative models for images: rationale‡Provides practical tools to specify the variables that determine the properties of images, either in terms of image features or scene properties. Important for characterizing the independent variables in psychophysics, and vision models‡Easier to characterize information flow: Mapping is is many-to-oneCan be used to model the likelihood of scene descriptions, p(I|S) (i.e. the probability of an image description given a scene description)Often easier to first specify p@I »SD and p@SD than p@S »ID.‡Characterize the knowledge required for inference mechanisms, e.g. neural networks for visual recognitionFeedforward procedures:4 7.ImageModelLinearSystems.nbPattern theory perspective: "analysis by synthesis" (Grenander, Mumford)‡Two basic concepts: Photometric (intensity, color) & geometric variation‡Two more basic concepts: Local and global representationsE.g. an edge can be locally represented in terms of the contrast and orientation at a point of an image. But a long edge (or contour) can also be represented by function with a small number of parameters (e.g. slope and intercept of straight line, or if curved, as a polynomial curve).An image can be represented in terms of a list of intensities at each location (local), or as we will see shortly, as a linear combination of global patterns (sine-wave gratings or other "basis functions").7.ImageModelLinearSystems.nb 5‡Two fundamental classes of image modeling: 1) image- (or appearance) based2) scene basedAs an example, image-based modeling tools are provided in software packages like: Adobe Photoshop or GIMP, or Adobe Illustrator For an introduction to image manipulation using Mathematica, see: http://library.wolfram.com/infocenter/Articles/1906/Scene-based modeling tools are provided in 3D graphics packages like: Maya, 3DS Studio Max, Sketchup, or software development packages like OpenGL.‡Sidenote: Ways of getting images into MathematicaExampleData@"TestImage"D8FileNameSetter@Dynamic@fDD, Dynamic@fD<: Browse… , f >colorpic = Import@fDcolorpic = Import@Experimental`FileBrowse@FalseDD;H*Undocumented*L‡Raster plotrcolorpic = colorpic@@1, 1DDê511.;Graphics@Raster@rcolorpicDD‡Gray scale plotgrayscalepic = MapB0.3 Ò@@1DD + 0.59 Ò@@2DD + 0.11 Ò@@3DD255&,N@colorpic@@1, 1DDD, 82<F;ArrayPlot@grayscalepic, DataReversed Ø True, ColorFunction Ø "Graytones"DImage-based modeling6 7.ImageModelLinearSystems.nbImage-based modelingFour classes of image-based models‡Linear intensity-based(Photometric)Basic idea is to represent an image I” in terms of linear combinations or sums or other images Ii:I” = m1*I1 + m2*I2 + m3*I3 + ...where mi's are scalars. You can think of each image as a point in an N-dimensional space, where N is the number of pixels, and the dimensions of the point correspond to the pixel intensities.In[14]:=Directory@DOut[14]=êUsersêkerstenIn[15]:=colorpic1 = Import@"256Palin.jpg"D;colorpic2 = Import@"256Biden.jpg"D;In[17]:=rcolorpic1 = colorpic1@@1, 1DDê511.;rcolorpic2 = colorpic2@@1, 1DDê511.;rcolorpic1 = colorpic1@@1, 1DDê511.;rcolorpic2 = colorpic2@@1,