IntroductionExampleMartingaleMarkov processGoal of the courseMATH 56A SPRING 2008STOCHASTIC PROCESSESKIYOSHI IGUSAThese are lecture notes for Math 56a, Spring Semester 2008. Sincecomplete notes from last year are available online, I will write thesenotes after each class so that they reflect what I actually said or meantto say. These notes are meant replace last year’s notes. However, theydo not replace the book: “Introduction to Stochastic Processes, 2nded.” by Greg Lawler. Please read the book with me so we can discussit. Here is the plan for the course:(1) st week (jan 17,23) Chap 0: Diff eq’s and linear recursion(2) nd week (jan 24,28,30,31) Chap 1: Finite Markov chains, Quiz(3) th week (feb 4,6,7) Chap 1: Finite Markov chains(4) th week (feb 11,13,14) Chap 2: Countable Markov chains(5) th week (feb 25,27,28) Chap 3: Continuous time Markov chains,Quiz(6) th week (mar 3,5,6) Ch 4: Stopping time(7) th week (mar 10,12,13) Ch 5: Martingales, Quiz(8) th week (mar 17,19,20) Ch 6: Renewal processes(9) th week (mar 24,26,27) Ch 7: Reversible Markov chains, Quiz(10) th week (mar 31, apr 2,3) Ch 8: Weiner process: This is wherethe intense stuff starts!(11) th week (apr 7,9,10) Ch 8: more(12) th week (apr 14,16,17) Ch 9: Stochastic integrals(13) th week (apr 28,30) Ch 9: more.The weeks don’t quite line up. I need the extra day (Jan 24th) atthe end. So, I’ll need to start each topic at the end of the previousweek.Date: January 17, 2008.12 KIYOSHI IGUSAIntroductionIn the first lecture, I discussed the concept of a stochastic processand gave a very quick introduction to two of the main concepts in thiscourse: martingales and Markov processes. I also tried to convey theflavor and philosophy of the course.A stochastic process is defined to be a random process which evolveswith time. For example, if you toss two dice then you get the numbers2 through 12 with a certain fixed probability distribution. This isstandard probability theory. An example of a stochastic process mightbe: Toss two dice and get a total of X1. Then toss that many dice andget a total of X2and so on. As time goes on you will need a lot of dice!Example. The next example I gave was the question: What is theprobability that your family name will survive? The answer I got was 0.I.e., with probability 1, everyone on Earth will have the same last name.This is the male version. You get your last name from your father. But,you get your mitochondria from your mother. The female version isthat everybody on Earth will eventually have the same mitochondriawhich is true!The setup for the population extinction problem (which we will studymore carefully later) is the following.Start at time t = 0 with a male population of N0.Ntis the male population after t generations.X1is the number of male offspring from the first man,X2is the number of male offspring from the second man, etc.Then, the number of males in the next generation will beN1= X1+ X2+ · · · + XN0.Assume that Xiare independent identically distributed (i.i.d.) randomvariables. In particular, they all have the same expected value:E(X1) = E(X2) = · · · = µ.So,E(N1) = µN0.This repeats and we getE(Nt) = µtN0.This is exponential growth.MATH 56A SPRING 2008 STOCHASTIC PROCESSES 3Martingale. One thing that is good to do is to make a “martingale”:Mt=Ntµt.Then,E(Mt) = N0.This is constant. That makes Mta martingale. (A martingale is astochastic random variable which you expect to have the same valuetomorrow as it has today.)The “Martingale Convergence Theorem” now tells us that Mtcon-verges to M∞. On page 119 of our book, it says that, if µ > 1 thenE(M∞) = E(M0) (= N0in this case). But, in class I argued that µ = 1and that the expected value of M∞will be zero!Markov process. For this I converted the problem into a Markovprocess. This is defined to be a system in which there is a fixed setof states and each state there is a fixed probability of going to eachother state. For example, in the random walk the states are the integerpoints on the real line. If you are at any point, the probability of goingto the left one space is 1/2 and the probability of going to the rightone space is 1/2.For the population problem, the states are: 0, 1, 2, 3, 4, · · · and youare in state Ntat time t. Given Ntmen in generation t, there is a certainprobability of every possible number of males in the next generation.So, we have a Markov process. We will learn that, in a Markov system,there are only two types of states: “recurrent” and “transient”. Arecurrent state is one that you keep coming back to with probabilityone. The only recurrent state is 0 (extinction). All other states mustbe transient which means you only go there a finite number of times.I will explain this later in the course.Since all finite states except 0 are transient, the Markov processwill “almost surely” (a.s.) go to 0 or ∞. Almost surely means “withprobability one.” I did not explain why infinity is not possible. In anycase, the answer I got was P(M∞) = 0, a.s.Goal of the course. We will look more carefully at this and otherexample. But the main example that I am interested in is the Black-Scholes equation. Some of you already know this equation from eco-nomics where it is usually derived using a binomial distribution. Wewill do a more serious analysis of this equation using stochastic inte-gration. Since this is the last topic in the book, we need to cover theentire book! We will go very fast, skipping some of the things at thebeginning so that we can get to the