57 IV. Linear systems analysis Linear systems analysis refers to a set of mathematical techniques that can be used to analyze and describe input/output systems that satisfy certain linearity assumptions. The inputs and outputs of linear systems are represented by functions. These functions can be of any number of variables, but for present purposes there will only be one variable, time (t) or space (x), or two variables, space-space (x,y), or space-time (x,t). For example the input functions might represent the variation in light level on a single receptor over time, the spatial distribution of light entering the eye, or the spatial distribution of neural activity transmitted from the eye to the brain along the optic nerve. The output functions might represent the electrical response of the receptor over time, the spatial distribution of light falling on the retina, or the spatial distribution of neural activity of a group of cells receiving inputs from the optic nerve. The letter L will be used to represent a linear system; i(t), i(x,y), etc. will represent input functions; and o(t), o(x,y), etc. will represent output functions. Thus, we can write in the one-dimensional case, o(t)=Li(t){} (4.1) or in the two-dimensional case, o(x,y)=Li(x,y){} (4.2) The curly brackets are used as a reminder that a linear system takes a whole function as input and gives a whole function as output. For example, the optical system of the eye can be regarded as a linear system that transforms the two-dimensional light distribution in an object plane, i(x,y), into a two-dimensional light distribution in the image plane, o(x, y). Figure 4.1 illustrates graphically a one-dimensional input function, a linear system, and a one dimensional output function. To make the figure concrete the reader might imagine that the left side represents a temporal signal (rectangular pulse) turned on shortly after time 0, while the right side represents the response of the linear system to this signal. A. Constraints defining linear systems In order for a one-dimensional system to be linear, it must satisfy the following constraint, Lai1(t)+bi2(t){}=aL i1(t){}+bL i2(t){} (4.3)58 where i1(t) and i2(t) are two arbitrary input functions and a and b are arbitrary constants (scalars). In words, the constraint is that the response of a linear system to the sum of two input function equals the sum of the responses to each input taken alone, and also scaling an input function by some amount scales the output function by exactly the same amount. In Figure 4.1B, another pulse has been added to the one in Figure 4.1A. If the system is linear then the response must be the sum of the individual responses (as shown). In Figure 4.1C, the input pulse in Figure 4.1A has been doubled in amplitude. If the system is linear the response should double in amplitude (as shown). Figure 4.1 The constraint is essentially the same for two-dimensional stimuli, Lai1(x,y)+bi2(x,y){}=aL i1(x,y){}+bL i2(x,y){} (4.4) where i1(x, y) and i2(x, y) are two arbitrary input functions and a and b are arbitrary constants (scalars). A somewhat more constrained type of system (the one that we will focus on) is the linear shift-invariant (LSI) system. Shift invariant means that shifting the input along the input dimensions shifts the output by a corresponding amount without changing the output shape. In other words, for a system to satisfy this constraint, a shift in the starting time or position of an input should not change the shape of the output. More formally, if59 o(t)=Li(t){} (4.5) then o(t−t0)=Li(t−t0){} (4.6) where t0 is an arbitrary shift along the t axis. In the two dimensional case, if, o(x,y)=Li(x,y){} (4.7) then o(x−x0,y−y0)=Li(x−x0,y−y0){} (4.8) where x0 and y0 are arbitrary shifts along the x-axis and y-axis, respectively. (Note that the linear system may perform, among other things, an arbitrary linear transformation of the axes.) Many systems that are not shift invariant globally can be approximated and analyzed as LSI systems in local regions. In Figure 4.1D the pulse in Figure 4.1A has been shifted in time. If the system is shift invariant the output should be shifted without a change in shape (as shown). Any system that satisfies the above constraints can be studied and characterized using the linear-systems techniques described below. However, the above constraints are very strong; thus, it is important to keep in mind that there are many systems that cannot be analyzed meaningfully with these techniques. Nonetheless, there a good number of situations where the constraints hold accurately and many others where they hold well enough for a linear-systems analysis to be useful. B. Impulse-response functions and convolution If a system is linear and shift invariant it can be completely characterized by its response to an instantaneous pulse of finite area (for one-dimensional systems) or of finite volume (for two-dimensional systems). This instantaneous pulse is known as an impulse function or delta function. The response to an impulse of unit area or volume is called the impulse response function. It is of course impossible to produce pulses of zero duration and an area or volume of one. However, for every physically realizable linear system, it is possible to select the pulse duration short enough so that the response is essentially identical to the impulse-response function. The use of impulse functions is a mathematical convenience.60 An impulse function is represented graphically as an arrow with height equal to its area or volume. The arrow on the left in Figure 4.2A represents an impulse function of unit area presented a location 0. The function labeled h(t) on the right represents an impulse-response function. The impulse-response function completely characterizes a LSI system because the response of the system to any input can be predicted from knowledge of the impulse-response function alone. This fact rests on the convolution formula, which will now be derived in an intuitive (not rigorous) fashion. Figure 4.2 1. Convolution formula for one-dimensional systems Consider an arbitrary input function, i(t), to a LSI system with impulse response function h(t). This arbitrary input is illustrated in Figure 4.2B. The first thing to note is that the input function can be broken down into the sum of short