2. Answers to Worksheet 2MATH 56A: STOCHASTIC PROCESSESWORKSHEET2. Answers to Worksheet 2P =.8 0 .2 0 00 .5 0 0 .5.3 0 .7 0 0.2 0 0 .6 .20 1 0 0 0(1) Calculate the rank of P − I5by column reduction.P − I =−.2 0 .2 0 00 −.5 0 0 .5.3 0 −.3 0 0.2 0 0 −.4 .20 1 0 0 −1The first step is to clear the last column. This is a column operation given by addingthe first 4 columns to the last column:−.2 0 .2 0 00 −.5 0 0 0.3 0 −.3 0 0.2 0 0 −.4 00 1 0 0 0Divide the fourth column by −.4 and use it to clear the fourth row:−.2 0 .2 0 00 −.5 0 0 0.3 0 −.3 0 00 0 0 1 00 1 0 0 0Now add the first column to the 3rd column:−.2 0 0 0 00 −.5 0 0 0.3 0 0 0 00 0 0 1 00 1 0 0 0This is good enough because each row has only one nonzero entry.1(2) How many recurrent classes does this Markov chain have?The rank of P − I is equal to 3 (the number of nonzero columns in the reducedform). Therefore, the nullity is 5 − 3 = 2. (This is the number of zero columns inthe reduced form.) So, there are exactly 2 recurrent classes.(3) Find the left null vectors of P −I5. Normalize to get the basic invariant distributions.The left null vectors are:(3, 0, 2, 0, 0), (0, 2, 0, 0, 1).These are the solutions of the equations(−.2)x1+ .3x3= 0(−.5)x2+ (1)x5= 0x4= 0which come from the matrix equation:(x1, x2, x3, x4, x5)−.2 0 0 0 00 −.5 0 0 0.3 0 0 0 00 0 0 1 00 1 0 0 0= (0, 0, 0, 0, 0).When you normalized these two vectors, you get the basic invariant probability dis-tributions:π1= (35, 0,25, 0, 0), π2= (0,23, 0, 0,13)(4) What are the supports of these vectors? These are the recurrent classes.The supports are the coordinates which are nonzero. These are:supp(π1) = {1, 3} = R1, supp(π2) = {2, 5} = R2.(5) Find all invariant distributions π.These are given by the formula:π = tπ1+ (1 − t)π2=3t5,2(1 − t)3,2t5, 0,1 − t3where 0 ≤ t ≤ 1.(6) If the initial distribution isα =0,13,13,13, 0,what is the invariant distributionαP∞:= limn→∞αPn?To answer this we need to determine t, the probability of ending up in recurrentstate R1. Since 2 and 3 are in R2, R1respectively, we start in each of those recurrentclasses with probability 1/3. For the remaining 1/3rd of the time we start in thetransient state 4. We need to look at the 4th row of the original matrix P to seewhat happens in that case:(.2, 0, 0, .6, .2).This show a loop at 4. When you escape that loop you go with equal probabilityto 1 and 5 which are in the recurrent classes R1, R2. So, you end up in R1, R2laterwith probability 1/6, 1/6 resp. In total, the probability is 1/2, 1/2 that you end upin R1, R2. So,t = P(Xn∈ R1for large n) =12.So,αP∞= π =310,13,15, 0,16.(7) Renumber the states and put the matrix P into canonical form.The recurrent states should come first. To avoid confusion, I will use letters to indicatethe reordering:a = 1, b = 3, c = 2, d = 5, e = 4.Then the matrix P becomes:P10 00 P20S1S2Q=.8 .2 0 0 0.3 .7 0 0 00 0 .5 .5 00 0 1 0 0.2 0 0 .2
View Full Document
Unlocking...