1. Homework 5 (Chap 3)3.5. 3.8. 3.11. 3.12.MATH 56A: FALL 2006HOMEWORK AND ANSWERSMath 56a: Homework 51. Homework 5 (Chap 3)p. 84 #3.5, 8, 11, 123.5. Let Xtbe a Markov chain with state space S = {1, 2} and rates α(1, 2) = 1, α(2, 1) = 4.Find Pt.The infinitesmal generator isA =−1 14 −4This matrix has eigenvalues 0, −5 with corresponding right eigenvectors (1, 1)t, (1, −4)tform-ing the matrix QQ =1 11 −4, D =0 00 −5, Q−1=4/5 1/51/5 −1/5Pt= etA= QetDQ−1=1 11 −41 00 e−5t4/5 1/51/5 −1/5So,Pt=154 + e−5t1 − e−5t4 − 4e−5t1 + 4e−5t3.8. The infinitesmal generator isA =−3 1 1 10 −3 2 11 2 −4 10 0 1 −1(a) Find the invariant distribution π.This is the solution of πA = 0 normalized so that the sum of the coordinates is 1:π =138(3, 7, 9, 19) ≈ (.079, .184, .237, .5)(b) If X0= 1 what is the expected amount of time until the first jump?8HW AND ANSWERS 2006 9The change rate is 3 times per unit time. So, the expected wait is 1/3 of a unit of time.(c) If X0= 1 what is the expected time until you reach state 4?You take˜A = A with the 4th row and 4th column deleted. Then you want to solve theequation˜Ab = −1or:−3 1 10 −3 21 2 −4b(1)b(2)b(3)=−1−1−1The solution is easy: b(1) = b(2) = b(3) = 1. So, b(x) = (expected time to get from x to 4)= 1 for x = 1, 2, 3.3.11. Xtis the birth-death process with λn= 1 + 1/(n + 1) and µn= 1. Is this positiverecurrent, null recurrent or transient?Sinceλn= 1 +1n + 1=n + 2n + 1the product collapses:λnλn−1· · · λ0=n + 2n + 1·n + 1n· · ·21= n + 2The sumXλnλn−1· · · λ0µn+1µn· · · µ1=Xn + 2 = ∞So, the process is not positive recurrent. Also,Xµn· · · µ1λn· · · λ1=X2n + 2= ∞So, the process is not transient. Thus, it must be null recurrent.What about λn= 1 − 1/(n + 2)?This is almost the same thing:λn= 1 −1n + 2=n + 1n + 2the product collapses again:λnλn−1· · · λ0=n + 1n + 2·nn + 1· · ·12=1n + 2The sumXλnλn−1· · · λ0µn+1µn· · · µ1=X1n + 2= ∞10 HW AND ANSWERS 2006So, the process is not positive recurrent. Also,Xµn· · · µ1λn· · · λ1=Xn + 22= ∞So, the process is not transient. Thus, it must be null recurrent.3.12. For λn= nλ, µn= nµ what values of λ, µ make extinction probability 1?In this problem we first have to make the Markov chain irreducible by changing λ0to be1. Then the extinction probability is one if and only if the new irreducible chain is recurrent,i.e., not transient. So, we take the sum:Xµn· · · µ1λn· · · λ1=XµnλnThis converges (making the chain transient) if and only if µ < λ. So, the extinction prob-ability is one if and only if µ ≥ λ and µ > 0. (When µ > λ the chain is positive recurrentand the expected time to extinction is
View Full Document
Unlocking...