# ISU STAT 432 - Formulas MarkovChains (2 pages)

Previewing page*1*of 2 page document

**View the full content.**## Formulas MarkovChains

Previewing page *1*
of
actual document.

**View the full content.**View Full Document

## Formulas MarkovChains

0 0 32 views

Other

- Pages:
- 2
- School:
- Iowa State University
- Course:
- Stat 432 - Applied Probability Models

**Unformatted text preview: **

Markov Chains A Markov chain is characterized by Pij Pr X n 1 j X n i X n 1 in 1 X 0 i0 Pr X n 1 j X n i Pij 0 i j 0 Pij 1 j 0 i 0 1 2 pi Pr X 0 i i 0 1 2 Pr X n j Pr X 0 i Pr X n j X 0 i i 0 pi Pij n i 0 Matrix notation The matrix of one step transition probabilities Pij The matrix of n step transition probabilities Pij n P n P P P P n times p p0 p1 p2 Pr X n j p P n P Q R 0 I The one step transition probability matrix partitioned into transient Q and absorbing R states N I Q 1 T N 1 The expected times spent in each state before being absorbed The expected times to absorption B N R The probabilities of absorption Limiting probabilities Regular chains for some n all elements of P n are 0 lim Pij n j n j N k Pkj k 0 j 0 1 2 N N j 1 j 0 Classification of states Accessible state j is accessible from state i if for some n 0 Pij n 0 Notation i j 1 Communication States i and j communicate if each state is accessible from the other Notation i j Two states that communicate are in the same class A Markov chain with only one state is irreducible Periodicity State i is said to have period d i if d i is the greatest common divisor of all integers n 1 for which Pii n 0 If d i 1 the chain is aperiodic Probability of first return to state i at step n given you start in state i f ii n Pr X n i X k i k 1 2 n 1 X 0 i Connection with n step probabilities Pii n n f ii k Pii n k k 0 Probability that a process starting in i returns to i at some time f ii n 0 f ii n lim N f ii n N n 0 Recurrent state i is recurrent if and only if f ii 1 or equivalently if and only if Pii n n 1 Transient state i is transient if and only if f ii 1 or equivalently if and only if Pii n n 1 Theorem For a recurrent irreducible aperiodic Markov chain 1 1 a lim Pii n b lim P ji n lim Pii n for all states j m n n n i nf ii n n 0 Stationary probability distribution A set of probabilities i for i 0 to that satisfy the following j k Pkj k 0 j 0 1 2 j 1 j 0 2

View Full Document