6. Differential structure of images"If I had more time, I would have written you a shorter letter", Pascal (1623-1662)6.1 The differential structure of imagesIn this chapter we will study the differential structure of discrete images in detail. This is thestructure described by the local multi-scale derivatives of the image. We start with thedevelopment of a toolkit for the definitions of heightlines, local coordinate systems andindependence of our choice of coordinates. << FrontEndVision`FEV`; Off@General::spellD;Show@Import@"Spiral CT abdomen.jpg"D, ImageSize -> 170D;Figure 6.1 An example of a need for segmentation: 3D rendering of a spiral CT acquisition ofthe abdomen of a patient with Leriche's syndrome (EuroRAD case #745, authors R. Brillo, A.Napoli, S. Vagnarelli, M. Vendola, M. Benedetti Valentini, 2000, www.eurorad.org).We will use the tools of differential geometry, a field designed for the structural descriptionof space and the lines, curves, surfaces etc. (a collection known as manifolds) that live there. We develop strategies for the generation of formulas for the detection of particular features,that detect special, semantically circumscribed, local meaningful structures (or properties) inthe image. Examples are edges, corners, T-junctions, monkey-saddles and many more. Wedevelop operational detectors in Mathematica for all features described.One can discriminate local and multi-local methods in image analysis. We specificallydiscuss here local methods, at a particular local neighborhood (pixel). In later chapters welook at multi-local methods, and enter the realm of how to connect local features, both bystudying similarity in properties with neighboring pixels ('perceptual grouping'), relationsover scale ('deep structure') and relations given by a particular model. We will discuss the useof the local features developed in this chapter into 'geometric reasoning'.91 6.1 The differential structure of imagesOne can discriminate local and multi-local methods in image analysis. We specificallydiscuss here local methods, at a particular local neighborhood (pixel). In later chapters welook at multi-local methods, and enter the realm of how to connect local features, both bystudying similarity in properties with neighboring pixels ('perceptual grouping'), relationsover scale ('deep structure') and relations given by a particular model. We will discuss the useof the local features developed in this chapter into 'geometric reasoning'.Why do we need to go in detail about local image derivatives? Combinations of derivativesinto expressions give nice feature detectors in images. It is well known that$%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%I∑LÅÅÅÅÅÅÅÅ∑xM2+ I∑LÅÅÅÅÅÅÅÅ∑yM2 is a good edge detector, and I∑LÅÅÅÅÅÅÅÅ∑yM2 ∑2LÅÅÅÅÅÅÅÅÅÅÅ∑x2- 2 ∑LÅÅÅÅÅÅÅÅ∑x ∑LÅÅÅÅÅÅÅÅ∑y ∑2LÅÅÅÅÅÅÅÅÅÅÅÅÅÅÅ∑x ∑y+ I∑LÅÅÅÅÅÅÅÅ∑xM2 ∑2LÅÅÅÅÅÅÅÅÅÅÅ∑y2is a good corner detector. But how do we come to such formula's? We can make an infinitenumber of such expressions. What constraints can/should we impose to come to a reasonablysmall set of basis descriptors? Is there such a basis? It turns out there is, and in this chapterwe will derive a formal complete set of such descriptive elements. A very important constraint in the development of tools for the description of image structureis to be independent of the choice of coordinates. We will discuss coordinatetransformations, like translations, rotations, zooming, in order to find a way to detect featuresinvariant to such coordinate transformations. In fact, we will discuss three 'languages' inwhich it is easy to develop a general strategy to come up with quite complex image structuredetectors: gauge coordinates, Cartesian tensors, and algebraic polynomial invariants. All these methodshave firm roots in mathematics, specifically differential geometry, and form an idealsubstrate for the true understanding of image structure.We denote the function that describes our landscape (the image) with LHx, yL throughout thisbook, where L is the physical property measured in the image. Examples of L are luminance,T1 or T2 relaxation time (for MRI images), linear X-ray absorption coefficient (for CTimages), depth (for range images) etc. In fact, it can be any scalar value. The coordinates x, yare discrete in our case, and denote the locations of the pixel. If the image is 3-dimensional,e.g. a stack of images from an MRI or CT scanner, we write LHx, y, zL. A scale-space ofimages, observed at a range of scales s is written as LHx, y; sL. We write a semicolon asseparator to highlight the fact that s is not just another spatial variable. If images are afunction of time as well, we write e.g. LHx, y, z; tL where t is the time parameter. In chapter17 we will develop scale-space theory for images sampled over time. In chapter 15 we studythe extra dimension of color in images and derive differential features in color-space, and inchapter 13 we derive methods for the extraction of motion, a vectorial property with amagnitude and a direction. We firstly focus on static, spatial images.6.2 Isophotes and flowlinesLines in the image connecting points of equal intensity are called isophotes. They are theheightlines of the intensity landscape when we consider the intensity as 'height'. Isophotes in2D images are curves, and in 3D surfaces, connecting points with equal luminance.(Greek: isos (isoV) = equal, photos (fotoV) = light): LHx, yL= constant orLHx, y, zL= constant. This definition however is for a continuous function. But the scale-space paradigm solves this: in discrete images isophotes exist because these are observedimages, and thus continuous (which means: infinitely differentiable, or C¶). Lines ofconstant value in 2D are Contours in Mathematica, which can be plotted withContourPlot. Figure 6.2 illustrates this for a blurred version of a 2D image.6. Differential structure of images 92(Greek: isos (isoV) = equal, photos (fotoV) = light): LHx, yL= constant orLHx, y, zL= constant. This definition however is for a continuous function.