Computational VisionU. Minn. Psy 5036Daniel KerstenLecture 7: Image Modeling, Linear Systems‡Initialize standard library files:Off@General::spell1D;OutlineLast time‡Bayesian decision theory applied to perceptionThe task is important: Some causes of image features are important to estimate (primary variables, Sprim), and some are not Ssec.Use the joint probability to characterize the "universe" of possibilities:(1)p ISprim, Ssec, IM(Directed) Graphs provide a way of modeling what random variables influences other random variables, and correspond-ingly how to decompose the joint into a simpler set of factors (conditional probabilities).Through conditioning on what is known, and integrating out what we don't care to estimate, we arrive at the posterior:(2)p ISprimIMAnd by Bayes' theorem the posterior probability is given by:p@SprimID =p@I SprimD p@SprimDp@IDp@I SprimD is the likelihood, and is determined by the generative model for Ip@SprimD is the prior model,which can also be considered part of the generative modelp@ID is fixedThe task determines how to use the posterior.Picking the most probable value of Sprim (MAP) results in the fewest errors on average‡Counter-intuitive consequences:‡Counter-intuitive consequences:Inference: Fruit classification example Pick most probable color--Answer "red"Pick most probable fruit--Answer "apple"Pick most probable fruit AND color--Answer "red tomato", not "red apple"Moral of the story: Optimal inference depends on the precise definition of the task‡Generalization to degrees of desired precision leads to Bayesian decision theorySlant estimation example illustrates how utility affects optimal estimatesImagine the image of an ellipse arrives at your retina, and for simplicity that somehow you know the width = 1. Your visual system measures the height of the ellipse in the image (x=1/2) and using this single number must estimate two object properties: 1) the aspect ratio of the actual disk causing the elliptical image, (i.e. the physical dimension of the disk orthogonal to the width); 2) the angle of inclination (slant) away from the vertical. 2 7.ImageModelLinearSystems.nbTodayGenerative modeling -- How can we characterize an image, or a collection of images?Introduction to modeling image structure: image-based vs. scene-basedLinear systems (linear image-intensity based modeling)Optical Resolution: Diffraction limits to spatial resolutionPoint spread function, convolutions and blurOverview of image modeling7.ImageModelLinearSystems.nb 3Overview of image modelingGenerative models for images: rationale‡Provides practical tools to specify the variables that determine the properties of images, either in terms of image features or scene properties. Important for characterizing the independent variables in psychophysics, and vision models‡Easier to characterize information flow: Mapping is is many-to-oneCan be used to model the likelihood of scene descriptions, p(I|S) (i.e. the probability of an image description given a scene description)Often easier to first specify p@I SD and p@SD than p@S ID.‡Characterize the knowledge required for inference mechanisms, e.g. neural networks for visual recognitionFeedforward procedures:Pattern theory perspective: "analysis by synthesis" (Grenander, Mumford)4 7.ImageModelLinearSystems.nb‡Two basic concepts: Photometric (intensity, color) & geometric variation‡Two more basic concepts: Local and global representationsE.g. an edge can be locally represented in terms of the contrast and orientation at a point of an image. But a long edge (or contour) can also be represented by function with a small number of parameters (e.g. slope and intercept of straight line, or if curved, as a polynomial curve).An image can be represented in terms of a list of intensities at each location (local), or as we will see shortly, as a linear combination of global patterns (sine-wave gratings or other "basis functions").‡Two fundamental classes of image modeling: 1) image- (or appearance) based2) scene basedAs an example, image-based modeling tools are provided in software packages like: Adobe Photoshop or GIMP, or Adobe Illustrator For an introduction to image manipulation using Mathematica, see: http://library.wolfram.com/infocenter/Articles/1906/Scene-based modeling tools are provided in 3D graphics packages like: Maya, 3DS Studio Max, Sketchup, or software development packages like OpenGL.Digression: Getting images into MathematicaThe easiest way is to drag an image from your computer desktop into the argument slot for Image[<drop it here>]7.ImageModelLinearSystems.nb 5In[26]:=ImageB FYou can find out the image type using ImageType[]. Use ImageDimensions[] to get the pixel dimensions of the raster. Often we'll want the raw pixel data in raster format:In[28]:=testimage = ImageDataB F;Remember if you execute ImageData[< >] without a semi-colon, Mathematica will output the whole array. We can use Dimension[] to see the dimensions of any list, including.Dimensions@testimageDOut[33]=8287, 200<It is a 256x256 pixel array with 3 values for each pixel, one for each color channel. If we want the gray-level versionIn[32]:=testimageg = ImageDataBColorConvertB , "Grayscale"FF;Check the Dimensions of testimageg.‡Mathematica databaseMathematica has a standard database library that includes images. You can get a list of testimages with: 6 7.ImageModelLinearSystems.nbIn[41]:=ExampleData@"TestImage"DOut[41]=88TestImage, Aerial<, 8TestImage, Aerial2<, 8TestImage, Airplane<,8TestImage, Airplane2<, 8TestImage, Airport<, 8TestImage, APC<,8TestImage, Boat<, 8TestImage, Bridge<, 8TestImage, CarAndAPC<,8TestImage, CarAndAPC2<, 8TestImage, ChemicalPlant<,8TestImage, Clock<, 8TestImage, Couple<, 8TestImage, Couple2<,8TestImage, Elaine<, 8TestImage, F16<, 8TestImage, Girl<,8TestImage, Girl2<, 8TestImage, Girl3<, 8TestImage, Gray21<,8TestImage, House<, 8TestImage, House2<, 8TestImage, JellyBeans<,8TestImage, JellyBeans2<, 8TestImage, Lena<, 8TestImage, Man<,8TestImage, Mandrill<, 8TestImage, Moon<, 8TestImage, Numbers<,8TestImage, Peppers<, 8TestImage, ResolutionChart<, 8TestImage, Ruler<,8TestImage, Sailboat<, 8TestImage, Splash<, 8TestImage, Tank<,8TestImage, Tank2<, 8TestImage, Tank3<, 8TestImage, TestPattern<,8TestImage, Tiffany<, 8TestImage, Tree<, 8TestImage, Truck<,8TestImage, TruckAndAPC<, 8TestImage, TruckAndAPC2<, 8TestImage, U2<<ExampleData[{"TestImage", "Peppers"}] can be used to display an image. The test image "Lena"