Math 19b: Linear Algebra with Probability Oliver Knill, Spring 2011Lecture 22: DistributionsA random variable is a function from a probability space Ω to the real line R. There are twoimpo r tant classes of random variables:1) For discrete random variables, the random variable X takes a discrete set of values. Thismeans that the random variable takes va lues xkand the proba bilities P[X = xk] = pkadd up to1.2) For continuous random variables, t here is a probability density function f(x) such thatP[X ∈ [a, b]] =Rbaf(x) dx andR∞−∞f(x) dx = 1.Discrete distributions1 We throw a dice and assume that each side appears with the same probability. The randomvariable X which gives the number of eyes satisfiesP[X = k ] =16.This is a discrete distribution called the uniform distribution on the set {1, 2, 3, 4, 5, 6 }.1234560.10.20.30.42 Throw n coins for which head appears with probability p. Let X denote the number ofheads. This random variable takes values 0, 1, 2, . . . , n andP[X = k ] = nk!pk(1 −p)n−k.This is the Binomial distribution we know.12345678910111213141516171819200.050.100.153 The probability distribution on N = {0, 1, 2, ... }P[X = k ] =λke−λk!is called the Poisson distribution. It is used to describe the number of radioactive decaysin a sample, the number of newborns with a certain defect. You show in the homework thatthe mean and standard deviation is λ Poisson distribution is the most important distributionon {0, 1, 2, 3, . . .}. It is a limiting case of Binomial distributions4 An epidemiology example fr om Cliffs notes: the UHS sees X=10 pneumonia cases eachwinter. Assuming independence and unchanged conditions, what is the probability of therebeing 20 cases of pneumonia this winter? We use the Poisson distribution with λ = 10 tosee P[X = 20] = 1020e−10/20! = 0.0018.The Poisson distribution is the n → ∞ limit of the binomial distribution if we chosefor each n the probability p such that λ = np is fixed.Pro of. Setting p = λ/n givesP[X = k] = nk!pk(1 −p)n−k=n!k!(n − k)!(λn)k(1 −λn)n−k= (n!nk(n − k)!)(λkk!)(1 −λn)n(1 −λn)k.This is a product of four factor s. The first factor n(n − 1)...(n − k + 1)/nkconverges to 1.Leave the second factor aλk/k! as it is. The third factor converges to e−λby the definitionof the exponential. The last factor (1 −λn)kconverges to 1 f or n → ∞ since k is kept fixed.We see that P[X = k] → λke−λ/k!.Continuous distributions5 A random variable is called a uniform distribution if P[X ∈ [a, b]] = (b − a) for all0 ≤ a < b ≤ b. We can realize this random variable on the probability space Ω = [a, b] withthe function X(x) = x, where P[I] is the length of an interval I. The uniform distributionis the most natural distribution on a finite interval.6 The random variable X on [0, 1 ] where P[[a, b]] = b − a is given by X(x) =√x. We haveP[X ∈ [a, b]] = P[√x ∈ [a, b]] = P[x ∈ [a2, b2]] = b2− a2.We have f (x) = 2x becauseRbaf(x) dx = x2|ba= b2−a2. The function f(x) is the probabilitydensity function of the random variable.ab7 A r andom variable with normal distribution with mean 1 and standard deviation 1 has theprobability densityf(x) =1√2πe−x2/2.This is an example of a continuous distribution. The nor ma l distribution is the most naturaldistribution on t he real line.ab8 The probability density functionf(x) = λe−λx.is called the exponential distribution. The exponential distribution is the most naturaldistribution on t he positive real line.Remark. The statements ”most natural” can be made mor e precise. Given a subset X ⊂ Rof the real line and the mean and standard deviation we can look at the distribution f on Xfor which the entropy −RXf log(f) dx is maximal. The uniform, exponential and normaldistributions extremize entropy on an interval, half line or real line. The reason why theyappear so often is that adding independent random variables increases entropy. If differentprocesses influence an experiment then t he entropy becomes large. Nature tries to maximizeentropy. Thats why these distributions are ”natural”.9 The distribution on the positive real axis with the density functionf(x) =1√2πx2e−(log(x)−m)22is called the log normal distribution with mean m. Examples of quantities which havelog normal distribution is the size of a living tissue like like length o r height of a po pula t ionor the size of cities. An other example is the blood pressure o f adult humans. A quantitywhich has a log normal distribution is a quantity which has a logarithm which is normallydistributed.Homework due March 30, 20111 a) Find the mean of the exponential distribution.b) Find the variance and standard deviation of the exponential distribution.c) Find the entropy −R∞0f(x) log(f(x)) dx in the case λ = 1.2 Here is a special case of the Students t distributionf(x) =2π(1 + x2)−2.a) Verify that it is a probability distribution.b) Find the mean. (No computation needed, just look at the symmetry).c) Find the standard deviation.To compute the integrals in a),c), you can of course use a computer algebra system ifneeded.3 a) Verify that the Poisson distribution is a probability distribution:P∞k=0P[X = k] = 1.b) Find the mean m =P∞k=0kP[X = k].c) Find the standard deviationP∞k=0(k − m)2P[X =