4/11/2007 1Lecture 10 (4.9.07)Image DescriptionShahram EbadollahiDIP ELEN E48304/11/2007 2Lecture Outline1 Motion (cont.) 1 Texture & Texture Quantification1 Moments1 Boundary Description1 Color Description1 MPEG-7 Descriptors4/11/2007 3Motion – object 1f2fobject motionε≤−= |),(),(|0),(21jifjififjidotherwise1Difference image:Cumulative difference imageabsolute positive negativeNo motion direction information !∑=−=nkkkcumjifjifajid11|),(),(|),(Tells us how often the image gray level was different from gray-level of reference image4/11/2007 4Motion Field• A velocity vector is assigned to each pixel in the image• Velocities due to relative motion between camera and the 3D scene• Image change due to motion during a time interval dt• Velocity field that represents 3-dimensional motion of object points across 2-dimensional imageMotion field4/11/2007 5Optical flow• Motion of brightness patterns in image sequence• Assumptions for computing optical flow:• Observed brightness of any object point is constant over time• Nearby points in the image plane move in a similar manner)(),,(),,(2∂+∂∂+∂∂+∂∂+=+++ Odttfdyyfdxxftyxfdttdyydxxf)(),,(2∂++++= Odtfdyfdxftyxftyxdtdyfdtdxfftyxfdttdyydxxfyxt+=−⇒=+++ ),,(),,(),(),( vudtdydtdxc ==cfvfuffyxt.∇=+=−Gray-level difference at same location over time is equivalent to product of spatial gray-level difference and velocityknown unknown4/11/2007 6Optical Flow Constraints• no spatial change in brightness, induce no temporal change in brightness 2 no discernible motion• motion perpendicular to local gradient induce no temporal change in brightness 2 no discernible motion• motion in direction of local gradient, induce temporal change in brightness 2 discernible motion• only motion in direction of local gradient induces temporal change in brightness and discernible motionvfuffyxt+=−Aperture problem4/11/2007 7Optical flow != Motion Field00=≠OFMF00≠=OFMF4/11/2007 8MPEG-7 Motion Descriptors1 Motion activity: slow-paced, fast-paced, action1 Features:1 Intensity of activity1 Direction of activity1 Spatial distribution of activity1 Temporal distribution of activity4/11/2007 9Texture - Definition4/11/2007 10Texture – Quantification Methods1 Statistical: compute local features at each point in image and derive a set of statistics from the distribution of local features1 1st, 2nd, and higher-order statistics based on how many points are used to define local features1 Structural: texture is considered to be composed of “texture elements”. Properties of the “texture elements” and their spatial placement rules characterizes the texture1 Original texture can be reconstructed from its structural description4/11/2007 11Statistical Texture Analysis1storder statisticsimage histogramfhf→• Obtain statistics of the histogram:Mean:Standard deviation:Skewness:Entropy:∑=niiih1)(∑=−niihi12)()(µ∑=−niihi13)()(µ∑=−niihih1)(log)(Measure of uniformity of histogram4/11/2007 12Statistical Texture Analysis1storder statisticsimagehistogramSkewness = 2.08Entropy = 0.88Skewness = 2.44Entropy = 0.77Skewness = -0.092Entropy = 0.97statistics4/11/2007 13Statistical Texture Analysis2ndorder statistics: Co-occurrence1 Joint gray-level histogram of pairs of pixels1 2D histograminmf=),(11jnmf=),(22θd12L1 2 Lij]),(,),(Pr[),(2211),(jnmfinmfjiPd==≈θ),(),(jiPdθ4/11/2007 14Statistical Texture Analysis2ndorder statistics: Co-occurrencejk°0°45°90°135|}),(,),(,||,0:)()()),(),,{((|),()0,(jnmfilkfdnlmkNMNMnmlkjiPd===−=−×××∈=°=θ|}),(,),(),,(),(:)()()),(),,{((|),()45,(jnmfilkfdnldmkdnldmkNMNMnmlkjiPd===−−==∨−=−=−×××∈=°=θ),()90,(jiPd°=θ),()135,(jiPd°=θ{}1is set cardinality4/11/2007 15Statistical Texture Analysis2ndorder statistics: Co-occurrence (example)image Co-occurrence matrix4/11/2007 16Statistical Texture Analysis2ndorder statistics: Co-occurrence (statistics)Angular 2ndmoment (energy):Maximum Probability:Contrast:Correlation:),(1 12),(jiPLiLjd∑∑= =θ),(max),(,jiPdjiθ),(log),(),(1 1),(jiPjiPdLiLjdθθ∑∑= =−Entropy:∑∑= =−LiLjdjiPji1 1),(),(λθκyxLiyxLjdjiijPσσµµθ∑∑= =−1 1),()],([∑ ∑= ==LiLjdxjiPi1 1),(),(θµ∑ ∑= =−=LiLjdxxjiPi1 1),(2),()(θµσ(measure of image homogeneity)(measure of local variations)(measure of image linearity)4/11/2007 17Statistical Texture Analysis2ndorder statistics: Difference StatisticsAngular 2ndmoment (energy):Mean:Contrast:)(102),(kPLkd∑−=θ∑−=10),()(LkdkkPθ∑−=−10),(),()(log)(LkddkPkPθθEntropy:∑−=10),(2)(LkdkPkθ∑=−∈=kjiLjiddjiPkP||},,1{,),(),(),()(1θθis a subset of co-occurrence matrix4/11/2007 18Statistical Texture Analysis2ndorder statistics: Fourier Power Spectrum),(),( vuFyxf↔2|),(|),( vuFvuP =∑==πθθ0),(2)( rPrP∑==2/0),()(LrrPPθθPower SpectrumuvuvIndicator for size of dominant texture element or texture coarsenessIndicator for the directionality of the texture4/11/2007 19Statistical Texture Analysis2ndorder statistics: Autocorrelation∑∑∑∑= =−=−=++−−=MkNlpMkqNlfflkfqlpkflkfqNpMMNqpC1 121 1),(),(),())((),(Large texture elements 3 autoccorrelation decreases slowly with increasing distanceSmall texture elements 3 autoccorrelation decreases rapidly with increasing distancePeriodic texture elements 3 periodic increase & decrease in autocorrelation value4/11/2007 20MPEG-7 Texture Descriptors1 Homogeneous texture descriptors1 Response of bank of filters sensitive to image scale and orientation1 Non-homogeneous texture descriptors (edge histogram)1 Spatial distribution of edges• vertical• horizontal• 45 degrees• 135 degrees• non-directional edgeimage4/11/2007 21Structural Texture4/11/2007 22Structural Texture4/11/2007 23Geometric Moments∫∫= dxdyyxfyxmqppq),((p+q)-th 2D geometric momentProjection of f(x,y) onto monomialqpyx• Why use moments?• Geometric moments of different orders represent spatial characteristics of the image intensity distribution00m0001000100//mmymmx==Total intensity of image. For binary image 2 area Intensity centroidbinary image 2 geometrical center4/11/2007 24Central Moments∫∫−−= dxdyyxfyyxxqppq),()()(00µ0000m=µ00110==µµ2002,µµ11µVariance about the centroidcovariance2]4)[()(2]4)[()(2/121120220022022/12112022002201µµµµµµµµµµ+−−+=+−++=IIPrincipal moment of inertia: θ)2(tan5.00220111µµµθ−=−2/10022/1001)/(2)/(2µµIbIa ==Image ellipse characterizes fundamental shape features and