174 BROWNIAN MOTION8.4. Brownian motion in Rdand the heat equation. The heatequation is a partial differential equation. We are going to convert itinto a probabilistic equation by reversing time. Then we can usingstopping time.8.4.1. definition.Definition 8.12. d-dimensional Brownian motion with drift µ = 0 ∈Rdand variance σ2is a vector valued stochastic pro cessXt∈ Rd, t ∈ [0, ∞)Xt= (X1t, X2t, ··· , Xdt)so that(1) Xtis continuous(2) The increments Xti−Xsiof Xton disjoint time intervals (si, ti]are independent.(3) Each coordinate of the increment is normal with the same vari-ance:Xit− Xis∼ N(0, σ2(t −s))and they are independent.This implies that the density function of Xt− Xsis a product ofnormal density functions:ft−s(x) = ft−s(x1) ···ft−s(xd) =d!i=11"2πσ2(t − s)e−x2i/2σ2(t−s)=1"2πσ2(t − s)de−||x||2/2σ2(t−s)since#ewhatever= ePwhateverandd$i=1x2i= ||x||2.Since this is the density of the increment Xt−Xs, it gives the transition“matrix”p∆t(x, y) = f∆t(y − x) =1√2πσ2∆tde−||y−x||2/2σ2∆tThe point is that ||y − x|| = ||x − y||. So,p∆t(x, y) = p∆t(y, x).In other words, Brownian motion (with zero drift) is a symmetric pro-cess. When you reverse time, it is the same. (It is also obvious that ifthere is a drift µ, the time reversed process will have drift −µ.)MATH 56A SPRING 2008 STOCHASTIC PROCESSES 1758.4.2. diffusion (the heat equation). If we have a large number of par-ticles moving independently according to Brownian motion then thedensity of particles at time t becomes a deterministic process calleddiffusion. It satisfies a differential equation called the heat equation.When we reverse time, we will get a probabilistic version of this equa-tion called the “backward equation.”Let f(x) b e the density of particles (or heat) at position x at timet. Then we have the Chapman-Kolmogorov equation, also called theforward equation:ft+∆t(y) =%Rdft(x)p∆t(x, y) dx&'()dx1···dxdBut, p∆t(x, y) = p∆t(y, x) since Brownian motion is symmetric whenµ = 0. So,ft+∆t(y) =%Rdft(x)p∆t(y, x) dxSince equations remain true when you change the names of the vari-ables, this equation will still hold if I switch x ↔ y. This gives thebackward equation:ft+∆t(x) =%Rdft(y)p∆t(x, y) dy&'( )This is an expected value.The RHS is an expected value since it is the sum of f(y) times itprobability. Since x moves to y in time ∆t, y = Xt+∆t.%Rdft(y)p∆t(x, y) dy = Ext(ft(Xt+∆t)) = E(ft(Xt+∆t) |Xt= x)Where Extmeans expectation is conditional on Xt= x. In words:F uture density at the present location x= expected value of the present density at the future location yusing the following interpretation of “present” and “future”time l ocationpresent t xfuture t + ∆t y176 BROWNIAN MOTION8.4.3. Calculate∂∂tft. I want to calculate∂∂tft(x) = lim∆t→0ft+∆t(x) − ft(x)∆t.Using the backward equation, this is= lim∆t→0Ext(ft(Xt+∆t)) − ft(x)∆tTo figure this out I used the Taylor series. Here it is when d = 1.ft(Xt+∆t) = ft(Xt) + f#t(Xt)∆X +12f##t(Xt)((∆X)2) + O((∆X)3)Here f#t=∂∂xft. The increment in X is∆X = Xt+∆t− Xt∼ N(0, σ2∆t)This means thatE((∆X)2) = σ2∆t.In other words, (∆X)2is expected to b e on the order of ∆t. So, (∆X)3is on the order of (∆t)3/2. So,E($)∆t=E(O((∆X)3)∆t→ 0 as ∆t → 0Taking expected value and substituting Xt= x we get:Ext(ft(Xt+∆t)) − ft(x) = f#t(x) E(∆X)&'( )=−µ=0+12f##t(x)Ext((∆X)2) + E($)=12f##t(x)σ2∆t + E($)Ext(ft(Xt+∆t)) − ft(x)∆t=12f##t(x)σ2+E($)∆t&'()→0So,∂∂tft(x) =σ22f##t(x)In higher dimensions we get the following∂∂tft(x) =σ22∆ft(x)where ∆ is the Laplacian:∆ =d$i=1∂2∂x2iMATH 56A SPRING 2008 STOCHASTIC PROCESSES 177This follows from the multivariable Taylor series:ft(Xt+∆t) = ft(Xt) +$i∂ft(Xt)∂xi∆Xit+12$i,j∂2ft(Xt)∂xi∂xj∆Xit∆Xjt+ $Since E(∆Xit) = −µi= 0, the Σiterms have expected value zero andthe Σi,jterms also have zero expected value when i (= j. This leavesthe Σi,iterms which give the Laplacian. I pointed out in class thatE(∆Xit) = −µibecause we are using the backward equation.8.4.4. boundary values. Now we want to solve the boundary valuedproblem, or at least convert it into a probability equation. Suppose wehave a bounded region B and we heat up the boundary ∂B.Letf(x) = current temperature at x ∈ Bg(y) = current temperature at y ∈ ∂BSuppose the g(y) is fixed for all y ∈ ∂B. This is the heating elementon the outside of your oven. The point x is in the inside of your oven.The temperature f(x) is changing according to the heat equation:∂f∂t=σ22∆f.We want to calculate u(t, x) = ft(x), the temperature at time t. Wealso want the equilibrium temperature v(x) = f∞(x). When the ovenhas been on for a while it stabilizes and∂∂tf∞(x) = 0which forces∆f∞(x) = 0If we use the backward equation we can use the stopping time T =the first time you hit ∂B. Taking it to be a stopping time means that178 BROWNIAN MOTIONthe boundary is “sticky” like flypaper. The particle x bounces aroundinside the region B until it hits the boundary ∂B and then it stops.(You can choose your stopping time to be anything that you want.)Then the backward equation, using OST, is:v(t, x) = Ex(g(XT)I(T ≤ t) + f(XT)I(t < T ))Here I(T ≤ t) is the indicator function for the event that T ≤ t.Multiplication by this indicator function is the same as the condition“if T ≤ t.” The equilibrium temperature is given byf∞(x) = v(x) = Ex(g(XT))This is an equation we studied before. v(x) is the value function. Itgives your expected payoff if you start at x and use the optimal strategy.g(x) is the payoff function. XTis the place that you will eventually stopif you use your optimal strategy which is the formula for the stoppingtime T .I gave one really simple example to illustrate this concept.Example 8.13. You give a professional gambler $x and send him toa casino to play until he loses (when he has $0) or wins (by getingy = $103). The gambler gets a fee of $a if he loses and $b if he wins.The question is: What is his expected payoff?T = stopping time is the first time that XT= 0 or y. We haveB = [0, y] with boundary ∂B = {0, y} and boundary values:g(0) = a, g(y ) = b.The expected payoff, starting at x, isv(x) = Ex(g(XT)) = aPx(XT= 0) + bPx(XT= y).But, Ex(XT) = X0= x by the Optimal Sampling Theorem. This is:yPx(XT= y) = xmaking Px(XT= y)