extbf {Last Time} extbf {Continuous Time Setup} extbf {Continuous Time Markov Process} extbf {Transition Probability Function} extbf {Homogeneous Markov Process} extbf {Initial Distribution} extbf {Existence of Markov Process} extbf {Stationary, Independent Increments} extbf {Brownian Motion} extbf {Functions of Markov Processes} extbf {Strong Markov Property}Last Time•Discrete time Markov chains•Transition probabilities•Chapman-Kolmogorov equations•Initial distribution•Strong Markov propertyToday’s lecture: Section 6.1MATH136/STAT219 Lecture 24, November 19, 2008 – p. 1/11Continuous Time Setup•Let S be a closed subset of IR and let B be thecorresponding Borel σ-field•Let {Xt, t ≥ 0} be a continuous time SP on (Ω, F, IP ) withstate space S, i.e. Xttakes values in S for all t•Let {Ft} be the canonical filtration of {Xt}(Ft= σ(Xs, 0 ≤ s ≤ t))MATH136/STAT219 Lecture 24, November 19, 2008 – p. 2/11Continuous Time Markov Process•The SP {Xt} is a (continuous time) Markov process if forall t, s ≥ 0 and any A ∈ BIP (Xt+s∈ A|Ft) = IP (Xt+s∈ A|Xt) a.s.•Equivalent condition: for all t, s ≥ 0 and any bounded,measurable function f : S 7→ IRIE[f(Xt+s)|Ft] = IE[f(Xt+s)|Xt] a.s.MATH136/STAT219 Lecture 24, November 19, 2008 – p. 3/11Transition Probability FunctionA function p : [0, ∞) × B × S 7→ [0, 1] is a (regular) stationarytransition probability functionif:•For any t ≥ 0 and x ∈ S, pt(·|x) is a probability measure on(S, B)•For any x ∈ S and A ∈ B, p0(A|x) = 1 if x ∈ A and 0otherwise•For all A ∈ B, p·(A|·) is Borel-measurable (in t and x)•Chapman-Kolmogorov Equations: for all t, s ≥ 0, A ∈ B, andx ∈ Spt+s(A|x) =ZSpt(A|y)ps(dy|x)MATH136/STAT219 Lecture 24, November 19, 2008 – p. 4/11Homogeneous Markov Process•Let p be a stationary transition probability function•A Markov process {Xt, t ≥ 0} has transition function p if forall t, s ≥ 0 and A ∈ B:IP (Xt+s∈ A|Ft) = ps(A|Xt) a.s.; that is,IP (Xt+s∈ A|Ft)(ω) = ps(A|Xt(ω)) a.s.Such a Markov process is(time) homogeneous.MATH136/STAT219 Lecture 24, November 19, 2008 – p. 5/11Initial Distribution•Let {Xt, t ≥ 0} be a homogeneous Markov process•The initial distribution of the process, denoted π, is thedistribution of X0, i.e.π(A) = IP (X0∈ A), A ∈ B•The distribution of Xtfor any t ≥ 0 is determined by theinitial distribution π and the transition probability function{pt(A|x) : t ≥ 0, A ∈ B, x ∈ S}•The initial distribution and the transition probability functiondetermine the FDD’s of the homogeneous Markov processMATH136/STAT219 Lecture 24, November 19, 2008 – p. 6/11Existence of Markov Process•Given (S, B), a probability measure π on (S, B), and astationary transition probability function p•There exists a probability space (Ω, F, IP ) and a SP{Xt, t ≥ 0} defined on it with state space S such that◦{Xt} is a homogeneous Markov process◦{Xt} has initial distribution π: IP (X0∈ A) = π(A), A ∈ B◦{Xt} has transition function pt(A|x):IP (Xt+s∈ A|Xt) = ps(A|Xt) for all t, s ≥ 0, A ∈ B•If the initial distribution satisfies π({x}) = 1 for some x ∈ S,we denote IP by IPx, i.e. IPx(X0= x) = 1MATH136/STAT219 Lecture 24, November 19, 2008 – p. 7/11Stationary, Independent Increments•Recall◦stationary increments:Xt+s− Xsd= Xt− X0for all s, t ≥ 0◦independent increments:Xt+s− Xsis independent of Fsfor all s, t ≥ 0•If a continuous time SP {Xt, t ≥ 0} has independentincrements then it is a Markov process•If a continuous time SP {Xt, t ≥ 0} has independent andstationary increments then it is a homogeneous MarkovprocessMATH136/STAT219 Lecture 24, November 19, 2008 – p. 8/11Brownian Motion•Brownian motion {Wt} is a homogeneous Markov processwith state space S = IR and transition probability functionpt(A|x) =ZA1√2πte−(y−x)22t, A ∈ B, x ∈ IRMATH136/STAT219 Lecture 24, November 19, 2008 – p. 9/11Functions of Markov Processes•Suppose f : [0, ∞) × S 7→ IR is a nonrandom function suchthat for each t ≥ 0 the function f (t, ·) : S 7→ IR is invertible•Suppose that g : [0, ∞) 7→ [0, ∞) is a nonrandom invertibleand strictly increasing function•If {Xt, t ≥ 0} is a Markov process•Then {f(t, Xg(t)), t ≥ 0} is a Markov process•If {Xt, t ≥ 0} is a homogeneous Markov process•Then {f(Xg(t)), t ≥ 0} is a homogeneous Markov processMATH136/STAT219 Lecture 24, November 19, 2008 – p. 10/11Strong Markov Property•Let {Xt, t ≥ 0} be a homogeneous Markov process withcanonical filtration {Ft}•The MP {Xt} has the strong Markov property if:IP (Xτ +s∈ A|Fτ) = IP (Xτ +s∈ A|Xτ)= IPXτ(Xs∈ A) a.s.,for all s ≥ 0, A ∈ B, and any {Ft}-stopping time with τ < ∞a.s.•A continuous time Markov process does not necessarilyhave the strong Markov property•However, the above equation does hold for any stoppingtime that takes at most countably many valuesMATH136/STAT219 Lecture 24, November 19, 2008 – p.