Last update: May 4, 2010VisionCMSC 421: Chapter 24CMSC 421: Chapter 24 1Outline♦ Perception generally♦ Image formation♦ Early vision♦ 2D → 3D♦ Object recognitionCMSC 421: Chapter 24 2Perception generallyStimulus (percept) S, World WS = g(W )E.g., g = “graphics.” Can we do vision as inverse graphics?W = g−1(S)CMSC 421: Chapter 24 3Perception generallyStimulus (percept) S, World WS = g(W )E.g., g = “graphics.” Can we do vision as inverse graphics?W = g−1(S)Problem: massive ambiguity!CMSC 421: Chapter 24 4Perception generallyStimulus (percept) S, World WS = g(W )E.g., g = “graphics.” Can we do vision as inverse graphics?W = g−1(S)Problem: massive ambiguity!CMSC 421: Chapter 24 5Perception generallyStimulus (percept) S, World WS = g(W )E.g., g = “graphics.” Can we do vision as inverse graphics?W = g−1(S)Problem: massive ambiguity!CMSC 421: Chapter 24 6ExampleCMSC 421: Chapter 24 7ExampleCMSC 421: Chapter 24 8Better approachesBayesian inference of world configurations:P (W |S) = α P (S|W )| {z }“graphics”P (W )| {z }“prior knowledge”Better still: no need to recover exact scene!Just extract information needed for– navigation– manipulation– recognition/identificationCMSC 421: Chapter 24 9Vision “subsystems”behaviorsscenesobjectsdepth mapoptical flowdisparityedges regionsfeaturesimage sequences.f. contourtrackingdata associationobjectrecognitionsegmentationfiltersedgedetectionmatchings.f.motions.f.shadings.f.stereoHIGH−LEVEL VISIONLOW−LEVEL VISIONVision requires combining multiple cuesCMSC 421: Chapter 24 10Image formationfimageplanepinholePP’YXZP is a point in the scene, with coordinates (X, Y, Z)P0is its image on the image plane, with coordinates (x, y, z)x =−fXZ, y =−fYZwhere f is a scaling factor.CMSC 421: Chapter 24 11ImagesCMSC 421: Chapter 24 12Images, continued195 209 221 235 249 251 254 255 250 241 247 248210 236 249 254 255 254 225 226 212 204 236 211164 172 180 192 241 251 255 255 255 255 235 190167 164 171 170 179 189 208 244 254 234162 167 166 169 169 170 176 185 196 232 249 254153 157 160 162 169 170 168 169 171 176 185 218126 135 143 147 156 157 160 166 167 171 168 170103 107 118 125 133 145 151 156 158 159 163 164095 095 097 101 115 124 132 142 117 122 124 161093 093 093 093 095 099 105 118 125 135 143 119093 093 093 093 093 093 095 097 101 109 119 132095 093 093 093 093 093 093 093 093 093 093 119255 251I(x, y, t) is the intensity at (x, y) at time ttypical digital camera ≈ 5 to 10 million pixelshuman eyes ≈ 240 million pixels; about 0.25 terabits/secCMSC 421: Chapter 24 13Color visionIntensity varies with frequency → infinite-dimensional signalintensityfrequencyfrequencysensitivityHuman eye has three types of color-sensitive cells;each integrates the signal ⇒ 3-element vector intensityCMSC 421: Chapter 24 14Primary colorsDid anyone ever tell you that the three primary colors are red, yellow, andblue?CMSC 421: Chapter 24 15Primary colorsDid anyone ever tell you that the three primary colors are red, yellow, andblue?Not correct.Did anyone ever tell you that• red, green, and blue are the primary colors of light, and• red, yellow, and blue are the primary pigments?CMSC 421: Chapter 24 16Primary colorsDid anyone ever tell you that the three primary colors are red, yellow, andblue?Not correct.Did anyone ever tell you that• red, green, and blue are the primary colors of light, and• red, yellow, and blue are the primary pigments?Only partially correct.What do we mean by “primary colors”?CMSC 421: Chapter 24 17Primary colorsDid anyone ever tell you that the three primary colors are red, yellow, andblue?They weren’t correct.Did anyone ever tell you that• red, green, and blue are the primary colors of light, and• red, yellow, and blue are the primary pigments?Only partially correct.What do we mean by “primary colors”?Ideally, a small set of colors from which wecan generate every color the eye can seeSome colors can’t be generated by any combination of red, yellow, and bluepigment.CMSC 421: Chapter 24 18Primary colorsUse color frequencies that stimulate the three color receptorsPrimary colors of light (additive)⇔ frequencies where the threecolor receptors are most sensitive⇔ red, green, bluePrimary pigments (subtractive)white minus red = cyanwhite minus green = magentawhite minus blue = yellow“Four color” (or CMYK) printing uses cyan, magenta, yellow, and black.CMSC 421: Chapter 24 19Edge detectionEdges in image ⇐ discontinuities in scene:1) depth2) surface orientation3) reflectance (surface markings)4) illumination (shadows, etc.)Edges correspond to abrupt changes in brightness or colorCMSC 421: Chapter 24 20Edge detectionEdges correspond to abrupt changes in brightness or colorE.g., a single row or column of an image:Ideally, we could detect abrupt changes by computing a derivative:In practice, this has some problems . . .CMSC 421: Chapter 24 21NoiseProblem: the image is noisy:The derivative looks like this:First need to smooth out the noiseCMSC 421: Chapter 24 22Smoothing out the noiseConvolution g = h ? f: for each point f[i, j], compute a weighted averageg[i, j] of the points near f[i, j]g[i, j] = Σku=−kΣkv=−kh[u, v]f[i − u, j − v]Use convolutionto smooth theimage, thendifferentiate:CMSC 421: Chapter 24 23Edge detectionFind all pixels at which the derivative is above some threshold:Label above-threshold pixels with edge orientationInfer “clean” line segments by combining edge pixels with same orientationCMSC 421: Chapter 24 24Inferring shapeCan use cues from prior knowledge to help infer shape:Shape from. . . Assumesmotion rigid bodies, continuous motionstereo solid, contiguous, non-repeating bodiestexture uniform textureshading uniform reflectancecontour minimum curvatureCMSC 421: Chapter 24 25Inferring shape from motionIs this a 3-d shape? If so, what shape is it?CMSC 421: Chapter 24 26Inferring shape from motionRotate the shape slightlyCMSC 421: Chapter 24 27Inferring shape from motionMatch the corresponding pixels in the two imagesUse this to infer 3-dimensional locations of the pixelsCMSC 421: Chapter 24 28Inferring shape using stereo visionCan do something similar using stereo visionPerceived objectRight imageLeft imageP0PCMSC 421: Chapter 24 29Inferring shape from textureIdea: assume actual texture is uniform, compute surface shape that wouldproduce this distortionSimilar idea works for shading—assume uniform reflectance,