**Unformatted text preview:**

EE 351K: PROBABILITY AND RANDOM PROCESSESSPRING 2020REVIEW NO. 3Ari ArapostathisThe University of Texas at AustinDecember 10, 20202Problem 1Let X0, X1, . . . , X10be iid exponential with parameter 2, and M a bino-mial RV with parameter 0.7. Assume independence.Compute EhPMi=0Xii.How about the variance? For this usevar(S) = varE[S |M]+ Evar(S |M) (1)with S =PMi=0Xi.Solution. Note that M takes values 0, 1, . . . , 10, so it is Binom(10, 0.7).We haveE[S |M] =M2, (2)andvar(S |M) =M4. (3)Therefore, we getE[S] = EE[S |M] =12E[M] =12× 10 × 0.7 = 3.5 .Also, using (1)–(3), we obtainvar S =14var(M) +14E[M]=1410 × 0.7 × 0.3 + 10 × 0.7=14× 10 ×0.7 × 1.3 =9.14= 2.275 .3Problem 2Let X1, X2, . . . , X6be iid exponential with parameter 2. Use Cheby-shev’s inequality to derive an upper bound for PP6i=1Xi> 5.Solution. Let S:=P6i=1Xi, and note that E[S] = 3, and var(S) =32.Thus we obtainP(S > 5) = P(S − 3 > 2)= P(|S − 3| > 2)≤var(S)4=38,where, in the second equality, we use the fact that S ≥ 0.4Problem 3Let X1, X2, . . . , X9be iid exponential with parameter 2. Use the CLT tocompute an approximate numerical value for PP9i=1Xi≤ 7.5.Solution. WriteP 9Xi=1Xi≤ 7.5!= P P9i=1(Xi− 0.5)√9 ×12≤3√9 ×12!≈ Φ(2) = 0.9772 (from standard Normal table)5Problem 4Let (X1, X2) be iid RVs with Xi∼ Exp(λ). Let Y = X1+ X2. Derivethe linear least squares estimator of X1given Y , that is, an estimator ofthe form X1= a + bY which minimizes the expected value of the squareerror.Solution. Note thatE[X1|Y ] = E[X2|Y ] = E[X3|Y ] .Since alsoE[X1+ X2|Y ] = Y ,these equations show thatE[X1|Y ] =Y2.Since this is linear in Y it matches the linear least squares estimator.6Problem 5X and Y are RVs taking only two distinct values with positive proba-bility. Suppose E[XY ] = E[X]E[Y ]. Is it the case that X⊥⊥Y ?Solution. Yes. By scaling, we can assume without loss of generalitythat X and Y are Bernoulli. Then, this is simple to show.7Problem 6A number of n balls are thrown at random into m boxes, with multipleoccupancy permitted. What is the expected number of empty boxes? Theanswer is(m−1)nmn−1.Solution. This is simple. For box i to be empty, all the balls have to bethrown in the remaining boxes. This has probabilitym−1mn. Since thereare m different boxes, the expected number ism ×m − 1mn=(m − 1)nmn−1.8Problem 7If X has the standard normal distribution, find the density function ofY = |X|, and the mean and variance of Y .Solution.fY(y) =2√2πe−y22for y ≥ 0 .For the mean, we calculateµY=Z∞0y2√2πe−y22dy =r2π.Concerning the variance, first computeE[Y2] =Z∞0y22√2πe−y22dy=12Z∞−∞y22√2πe−y22dy= 1 .Therefore,var(Y ) = 1 −2π.9Problem 8Let X1, X2, . . . , Xnbe iid with CDF F and PDF f. Express the jointPDF of U = min(X1, . . . , Xn) and V = max(X1, . . . , Xn) using f andF . Answer:fU,V(u, v) = n(n − 1)f (u)f (v)F (v) − F (u) n−2for u < v .Solution. First writeP(U > u, V ≤ v) =nYi=1P(u < Xi≤ v) =F (v) − F (u) nfor u < v.Then note thatfU,V(u, v) = −∂∂u∂∂vP(U > u, V ≤ v)= n(n − 1)f (u)f (v)F (v) − F (u) n−2.10Problem 9We are givenfX,Y(x, y) =13(x + 2y)e−x−yif x, y ≥ 0 .Find the marginal density of Y and the conditional density of X givenY = y. Show that P(Y > X) =712.Solution. An easy calculation shows thatfY(y) =13(2y + 1)e−yfor y ≥ 0 .Next we use the identityZy0xe−xdx = 1 − e−y− ye−y,to getZy013(x + 2y)e−x−ydx =131 − e−y− ye−ye−y+23y1 − e−ye−y.Thus integrating term by term we obtainP(Y > X) =Z∞0h131 − e−y− ye−ye−y+23y1 − e−ye−yidy=13−16−112+23−212=712.11Problem 10We are given fX(x) = xe−x22for x ≥ 0, and that Y is uniformly dis-tributed on [−ε, ε]. Let Z = X + Y . Find fZ(z). Then show thatP(Z > ε) =12εZ2ε0e−t2tdt .Solution. We start withfZ(z) =Z∞−∞fY(y)fX(z − y) dy=Zε−ε12ε(z − y)e−(z−y)22dy=12εZz+εz−εue−u22du=12εhe−(z−ε)22− e−(z+ε)22i.Therefore,P(Z > ε) =12εZ∞εhe−(z−ε)22− e−(z+ε)22idz=12εZ2εεe−(z−ε)22dz=12εZ2ε0e−t2tdt .12Problem 11Let X1, X2, . . . be iid continuous RVs. Define N as the unique indexsuch thatX1≥ X2≥ ··· ≥ XN−1and XN−1< XN.Show that P(N = k) =k−1k!for k = 2, 3, . . . , and that E[N] = e.Solution. Consider the events A1, A2, . . . , Ak−1defined byAj:=X1> X2> ··· > Xj−1> Xk> Xj> ···Xk−1}.It is clear that P(Aj) =1k!by symmetry. Therefore, since these events aredisjoint, we haveP(X1> X2> ··· > Xk−1, and Xk> Xk−1) =k−1Xi=1P(Ai) =k − 1k!.Next, note that∞Xk=2kk − 1k!=∞Xk=21(k − 2)!= e .13Problem 12Let X be a continuous RV which takes values in the interval [−M, M].Show thatP|X| ≥ a≥E|X| − aM −afor any a such that 0 ≤ a ≤ M.Hint: try instead showing the equivalent inequality:P|X| ≤ a≤EM −|X| M −aSolution. WriteEM −|X| =ZM−MM −|x|fX(x) dx≥Za−aM −|x|fX(x) dx≥M −|a|Za−afX(x) dx≥M −|a|P|X| ≤ a.14Problem 13Suppose X and Y are iid with mean 0 and variance 1, and in additionsuppose that (X + Y )⊥⊥(X − Y ). Show that X and Y are necessarilystandard Normal.Solution. Let the moment generating function of X (or Y ) be denotedby M(s). By independenceM(2s) = Ee2sX = Ees(X+Y )+s(X−Y ) = Ees(X+Y ) × Ees(X−Y ) = M(s) × M(s) × M(s) × M(−s)=M(s)3M(−s) .(4)Letting ψ(s):=M(s)M(−s), we obtain from (4) thatψ(s) =ψs22= ··· =ψs2n2nfor any n ≥ 1 . (5)Since µX= 0 and var(X) = 1, from the Taylor expansion we haveM(s) = 1 +12s2+ o(s2) . (6)Thus, by (6), the definition of ψ, and (5), we haveψ(s) = 1 + o(s2) =h1 + os222ni2n−−−→n→∞1 .This means that M(s) = M(−s), and so (4) and (6) giveM(s) = 1 +12s2+ o(s2) =h1 +12s222n+ os222ni4n−−−→n→∞es22.This shows that X is standard

View Full Document