MATH 56A: STOCHASTIC PROCESSESCHAPTER 66. RenewalMathematically, renewal refers to a continuous time stochastic pro-cess with states 0, 1, 2, . · · ·Nt∈ {0, 1, 2, 3, · · · }so that you only have jumps from x to x + 1 and the probability ofjumping from x to x + 1 depends only on how long the process was atstate x. Renewal occurs at each jump.Nt:= number of jumps that occur in time interval (0, t]The jumps (renewals) occur at times Y, Y + T1, Y + T1+ T2, etc. andY + T1+ · · · + Tn= inf{t | Nt= n + 1}The interpretation is that there is an object or process which lastsfor a certain amount of time which is random. When the object diesor the process stops then you replace the object with a new one or yourestart the process from the beginning: You “renew” the pro ces s eachtime it stops. The number Ntis equal to the number of times renewalhas taken place up to time t. Y is the lifetime of the initial process,T1is the lifetime of the second one, T2is the lifetime of the third one,etc. If the first process starts from the beginning then Y = 0 and thenumbering is different. Tnbecomes the lifetime of the nth process:T1+ · · · + Tn= inf{t | Nt= n}I gave a light bulb as an example. There are three kinds of lightbulbs:(1) The guaranteed light bulb which will last exactly 1000 hours.(2) The Poisson light bulb. This light bulb is as good as new aslong as it is working. Assume it has an expected life of 1000hours. (λ = 1/1000).(3) A general light bulb which has a general probability distributionwith the property that its expected life is 1000 hours.Date: November 14, 2006.12 MATH 56A: STOCHASTIC PROCESSES CHAPTER 6In all three cases,µ = E(T ) = 1000where T is the length of time that the light bulb lasts.The first question is: Which light bulb is worth more? The answeris that they are all worth the same. They all give an expected utilityof 1000 hours of light. With the general light bulb, there is anotherquestion: How long do you expect the last light bulb to last after ithas been used for a certain amount of time? This depends on the lightbulb. For example, if the guaranteed light bulb has been used for 500hours then it is only worth half as much as a new one. If the Poissonlight bulb lasts 500 hours then it is still worth the same as a new one.We w ill look at the value of a general light bulb (or a nenewal processwith a general distribution.)6.1. Renewal theorem. The guaranteed light bulb is an example ofa periodic renewal process. Each renewal occurs at multiples of 1000hours.Definition 6.1. A renewal process is periodic if renewals always occurat (random) integer multiples of a fixed time interval ∆t starting withthe first renewal which occurs at time Y .The renewal theorem says that, if renewal is not periodic, then theoccurrences of the renewal will be spread out evenly around the clock.The probability that it will occur will depend only on the length oftime you wait. Since the average waiting time is µ, the probability isapproximately the proportion of µ that you wait: P∼=∆t/µ.For the lightbulb, suppose you install a million lightbulbs at the sametime. Then after a while the number of light bulbs that burn out eachday will be constant. This (after dividing by one million) will be theequilibrium distribution.Theorem 6.2 (Renewal Theorem). If a renewal process is aperiodicthen, as t → ∞,P(renewal occurs in time (t, t + dt]) →dtµwhere µ = E(T ). This is equivalent to saying thatlimt→∞E(number of renewals in time (t, t + s]) =sµlimt→∞E(Nt+s− Nt) =sµMATH 56A: STOCHASTIC PROCESSES CHAPTER 6 36.2. age of current process. At any time t, let Atbe the life of thecurrent process. This would be the answer to the question: How longago did you replace the light bulb? The book says that the pair (Nt, At)determines the future of the process. Btdenotes the remaining life ofthe current process. (How long will the current light bulb last?) FirstI needed the following lemma.6.2.1. picture for an expected value.Figure 1. The shaded area ab ove the distribution func-tion F (t) is equal to the expectation.Lemma 6.3. If T ≥ 0 is a nonnegative random variable then theexpected value of T is given byE(T ) =!∞01 − F (t) dtProof. The expected value of T is defined by the integralE(T ) =!∞0tF (t) dtSubstituting the integralt =!t0ds =!0≤s≤tdswe get:E(T ) =!!0≤s≤tf(t) dsdtOn the other hand,1 − F (s) = P(T > s) =!∞sf(t) dt4 MATH 56A: STOCHASTIC PROCESSES CHAPTER 6So,!∞01 − F (s) ds =!∞0!∞sf(t) dtds =!!0≤s≤tf(t) dsdt = E(T )!6.2.2. distribution of current age. What is the density function for thecurrent age Atfor large t? I.e., what is P(s < At≤ s + ∆s)? This isgiven byP(s < At≤ s + ∆s)∼=t→∞∆sµ(1 − F (s))because: The renewal event must occur in a time interval ∆s: By therenewal theorem this has probability approximately ∆s/µ. The n thenext renewal event must occur at some time greater than s. That hasprobability 1−F (s) where F (s) is the distribution function of the lengthof time that each renewal process lasts. This is an approximation forlarge t which depends on the Renewal Theorem. See the figure.XFigure 2. The age of the current process tells when wasthe last renewal.To get the density function of the current age function Atwe haveto take the limit as ∆s → 0:ψA(s) = lim∆s→01∆sP(s < At≤ s + ∆s) =1 − F (s)µThe lemma says that the integral of this density function is 1 (as itshould be):!∞0ψA(s) ds =!1 − F (s)µds =µµ= 1MATH 56A: STOCHASTIC PROCESSES CHAPTER 6 5For the case of the exponential distribution we have 1 − F (t) = e−λtand µ = 1/λ. Sof(t) = λe−λt= λ(1 − F (t)) =1 − F (t)µ= ψA(t)and the age of the current process has the same distribution as theentire lifespan of the process.6.2.3. distribution of remaining life. The remaining life or residual lifeof the process at time t is simply how long we have to wait for the nextrenewal. It is called Bt. It is a little more complicated to analyze.XXFigure 3. The residual life Btseems to depend on thetime of the last renewal.We want to calculate the distribution function ΨB(x) = P(Bt≤ x). Inorder for this even to occur, we first need the last renewal to occur insome interval ds before t. This has probability ds/µ. Then we needthe event not to occur again during the time interval s before time tbut we need it to occur sometime in the time interval x after time t.This occurs with probability F (s + x) − F (s). But s ≥ 0