The Hessian, Taylor’s Theorem, Extrema, LagrangeMultipliers and Quadratic Forms∗Jonathan LeeNovember 20, 2008†An online version of these notes is posted on my website athttp://math.stanford.edu/~jlee/math51/.Differentials and Taylor’s Theorem• given a k-times differentiable function f : R → R, we can define its k-th order Taylorpolynomial at the point p to bepk(x) =kXi=0f(i)(p)i!· (x − p)i• Taylor’s theorem provides us an error estimate: if f : R → R is a (k + 1)-timesdifferentiable function, then there exists some number ξ between p and x such thatRk(x, a) = f(x) − pk(x) =f(k+1)(ξ)(k + 1)!· (x − a)k+1• given a differentiable function f : Rn→ R, we can define its first-order Taylor polyno-mial at the point p = (p1, . . . , pn) to bep1(x) = f(p) + Df(p) · (x − p)= f(p) +nXi=1∂f∂xi(p) · (xi− pi) ;similarly, if f is twice-differentiable, we can define its second-order Taylor polynomialat the point p to bep2(x) = f(p) + Df(p) · (x − p) +12(x − p)T· Hf(p) · (x − p) = f(p) +nXi=1∂f∂xi(p) · (xi− pi) +12nXi,j=1∂2f∂xixj(p) · (xi− pi)(xj− pj) ,∗Alternatively, All you ever wanted to know about vector calculus†Math goes well with turkey — have some of both over the break!1where for notational sanity, we define the Hessian matrix to be the n× n matrix whose(i, j)-th entry is∂2f∂xixj– find the first- and second-order Taylor polynomials forthe function at the pointf(x, y) = 1/(x2+ y2+ 1) a = (0, 0)f(x, y) = 1/(x2+ y2+ 1) a = (1, −1)f(x, y) = e2xcos(3y) a = (0, π)• as before, Taylor’s theorem provides an error estimate; the same formula holds, withthe change that ξ is taken to be a point on the line segment connecting p and xQuadratic Forms• given an n × n symmetric matrix, we can define a quadratic form Q : Rn→ R suchthat x 7→ xTAx• a quadratic form is defined to be positive definite, positive semi-definite, negative defi-nite, or negative semi-definite if the values it takes are positive, non-negative, negativeor non-p ositive, respectively; if none of these holds, the form is defined to be indefinite• Remark: by definition, if a quadratic form is positive definite, then it is also positivesemi-definite• Proposition: given a quadratic form Q : R2→ R defined by x 7→ xTAx, we canrecognize its type according to the following table:type of quadratic form Q eigenvalues of A A’s determinant A’s tracepositive definite all positive positive positivepositive semi-definiteall non-negative zero positivenegative definite all negative positive negativenegative semi-definite all non-positive zero negativeindefinite one positive, one negative negative irrelevantdegenerate both zero zero zeroExtrema of Functions• given a function f : Rn→ R, know what its extrema, both global and local, are definedto be; this (and the following) works analogously to the single-variable case• define a point p to be a critical point of a differentiable function f : Rn→ R ifDf(p) = 0• Theorem: lo cal extrema of differentiable functions must be critical points2• Theorem: let p be a f : Rn→ R be a twice-differentiable function; thenif Hf(p) isthen p is a . . . of fpositive definite local minimumnegative definite local maximumneither of the above but still invertible saddle point– find the point on the plane 3x − 4y − z = 24 closest to the origin– determine the absolute extrema off(x, y) = x2+ xy + y2− 6yon the rectangle given by x ∈ [−3, 3] and y ∈ [0, 5]– determine the absolute extrema off(x, y, z) = exp(1 − x2− y2+ 2y − z2− 4z)on the ball{(x, y, z) ∈ R3: x2+ y2− 2y + z2+ 4z ≤ 0}• by Heine-Borel, say that a subset X of Rnis compact if it is closed and bounded• Extreme Value Theorem: any continuous R-valued function on a compact topologicalspace attains global minima and maximaLagrange Multipliers• supposing that we have a continuously differentiable function f : S → R, where S ⊂ Rnis defined to be the set of solutions to g1= g2= · · · = gk= 0 for some continuouslydifferentiable functions g1, . . . , gk: Rn→ R, then we can determine the critical pointsof f by:– solving the system of linear equations (in the variables x, λ1, . . . , λk)Df(x) = λ1Dg1(x) + · · · + λkDgk(x) and g1(x) = g2(x) = · · · = gk(x) = 0using elimination, cross-multiplication or other convenient methods; each solutionx will be a critical point of f– determining the points x where the functions Dg1, . . . , Dgkare linearly dependent,which in the case k = 1 (and Dg = Dg1) amounts simply to finding those x suchthat Dg(x) = 0; only some of these points will be critical points and thus theyall need to be inspected individually• as a result, we now have three methods for determining the critical points of a contin-uously differentiable function f : S → R, for some specified set S:3– if S is a curve (which has dimension 1), a surface (which has dimension 2), or ingeneral, some subset of dimension n, then attempt to parametrize S by a functiong : Rn→ S and subsequently compute the critical points of f ◦ g : Rn→ R —this method can quite often be unnecessarily brutal, requiring many error-pronecalculations, which you may illustrate to yourself by finding the closest point ona line to a given point– find some geometric interpretation of the problem, draw some picture making itclear where the extrema should occur, and be creative computing the coordinatesof such extrema — for example, in the case of finding the closest point on a givenplane to a given point, this amounts to de termining the intersection of the planewith the unique line perpendicular to the plane that crosses the specified point– use Lagrange multipliers• good, wholesome, enriching entertainment:– find the largest possible sphere, centered around the origin, that can be inscribedinside the ellipsoid 3x2+ 2y2+ z2= 6– the intersection of the planes x − 2y + 3z = 8 and 2z − y = 3 forms a line; findthe point on this line closest to the point (2, 5, −1)– the intersection of the paraboloid z = x2+ y2with the plane x + y + 2z = 2 formsan ellipse; determine the highest and lowest (with respect to the z-coordinate)points on