6.041 Fall08 Quiz II Review11 Probability Density Functions (PDF)For a continuous RV X with PDF fX(x) (≥ 0),P(a ≤ X ≤ b) =ZbafX(x)dxP(x ≤ X ≤ x + δ) ≈ fX(x).δP(X ∈ A) =ZAfX(x)dxRemarks:- if X is continuous, P(X = x) = 0 ∀x!!- fX(x) may take values larger than 1.Normalization property:Z∞−∞fX(x)dx = 122 Mean and variance of a continuous RVE[X] =Z∞−∞xfX(x)dxE[g(X)] =Z∞−∞g(x)fX(x)dxVar(X) =Z∞−∞(x − E[X])2fX(x)dx= E[X2] − (E[X])2(≥ 0)E[aX + b] = aE[X] + bVar(aX + b) = a2Var(X)33 Cumulative Distribution FunctionsDefinition :FX(x) = P(X ≤ x)monotonically increasing from 0 (at −∞) to 1 (at +∞).• Continuous RV:FX(x) = P(X ≤ x) =Zx−∞fX(t)dt (continuous)fX(x) =dFXdx(x)• Discrete RV:FX(x) = P(X ≤ x) =Xk≤xpX(k) (piecewise constant)pX(k) = FX(k) − FX(k − 1)44 Normal/Gaussian Random VariablesStandard Normal RV: N(0, 1):fX(x) =1√2πe−x2/2E[X] = 0, Var(X) = 1General normal RV: N(µ, σ2):fX(x) =1σ√2πe−(x−µ)2/2σ2E[X] = µ, Var (X) = σ25• if Y = aX + b, then Y ∼ N(aµ + b, a2σ2).• CDF for standard normal φ(.) can be read in a table.• To evaluate CDF of a general standard normal, express it as afunction of a standard normal:X ∼ N(µ, σ2) ⇔X − µσ∼ N(0, 1)P(X ≤ x) = PX − µσ≤x − µσ= φx − µσwhere φ(.) denotes the CDF of a standard normal.65 Joint PDFJoint PDF of two cont i nuous RV X and Y : fX, Y(x, y).P(x ≤ X ≤ x + δ, y ≤ Y ≤ y + δ) ≈ fX, Y(x, y).δ2P(A) =Z ZAfX, Y(x, y)dxdyE[g(X, Y )] =Z∞−∞Z∞−∞g(x, y)fX, Y(x, y)dxdyfX(x) =Z∞−∞fX, Y(x, y)dyBy d efinition,X, Y independent ⇔ fX, Y(x, y) = fX(x)fY(y)76 Conditioning on an eventX a continuous RV, A a subset of the real linefX|A(x) =fX(x)P(X∈A)if x ∈ A0 otherwiseP(X ∈ B|X ∈ A) =ZBfX|A(x)dxE[X|A] =Z∞−∞xfX|A(x)dxE[g(X)|A] =Z∞−∞g(x)fX|A(x)dx8If A1, . . . , Anare disjoint events that form a partition of th e s amplespace,fX(x) =nXi=1P(Ai)fX|Ai(x) (≈ total probability theorem)E[X] =nXi=1P(Ai)E[X|Ai] (total expectation theorem)E[g(X)] =nXi=1P(Ai)E[g(X)|Ai]97 Conditioning on a RVX, Y continuous RV, A an event.P(x ≤ X ≤ x + δ|Y ≈ y) ≈ fX|Y(x|y).δfX|Y(x|y) =fX, Y(x, y)fY(y)fX(x) =Z∞−∞fY(y)fX|Y(x|y)dyP(A) =Z∞−∞P(A|X = x)fX(x)dx10E[Y |X = x] =Z∞−∞yfY | X(y|x)dyE[g(Y )|X = x] =Z∞−∞g(y)fY | X(y|x)dyE[g(X, Y )|X = x] =Z∞−∞g(x, y)fY | X(y|x)dyE[Y ] =Z∞−∞E[Y |X = x]fX(x)dxE[g(Y )] =Z∞−∞E[g(Y )|X = x]fX(x)dxE[g(X, Y )] =Z∞−∞E[g(X, Y )|X = x]fX(x)dx118 Continuous Bayes’ RuleX, Y continu ous RV, N discrete RV, A an event.fX|Y(x|y) =fY |X(y|x)fX(x)fY(y)=fY |X(y|x)fX(x)R∞−∞fY |X(y|t)fX(t)dtP(A|Y = y) =P(A)fY |A(y)fY(y)=P(A)fY | A(y)fY |A(y)P(A) + fY |Ac(y)P(Ac)P(N = n|Y = y) =pN(n)fY |N(y|n)fY(y)=pN(n)fY |N(y|n)PipN(i)fY |N(y|i)129 Independence of continuous RVX, Y independent ⇔ fX|Y(x|y) = fX(x)⇒ g(X), h(Y ) independent⇒ E[XY ] = E[X]E[Y ]⇒ E[g(X)h(Y )] = E[g(X)]E[h(Y )]⇒ Var(X + Y ) = Var(X) + Var(Y )1310 Derived distributionsDef: PDF of a function of a RV X with known PDF: Y = g(X).Method:• Get the CDF:FY(y) = P(Y ≤ y) = P(g(X) ≤ y) =Zx|g(x)≤yfX(x)dx• Differentiate: fY(y) =dFYdy(y)1411 ConvolutionW = X + Y , with X, Y independent.• Discrete case:pW(w) =XxpX(x)pY(w − x)• Continuous c ase:fW(w) =Z∞−∞fX(x)fY(w − x) dx15Mechanic s:- put the PMFs ( or PD Fs) on top of each other- flip the PMF (or PD F) of Y- shift th e flipped PMF (or PD F) of Y by w- cross-mul ti ply an d ad d (or evaluate the integral)In p artic ular, if X, Y are independent and normal, then• W = X + Y is normal• fX|W(x|w) is a normal PDF for any given w.1612 Law of iterated expectationsE[X|Y ] is a random variable that is a function of Y(the expe ctation is taken with respect to X).To compu te E[X|Y ], first express E[X|Y = y] as a function of y.Law of iterated e x pectations:E[X] = E[E[X|Y ]](equality betwe en two real numbers)1713 Law of conditional variancesVar(X|Y ) is a random variable that is a functi on of Y(the variance is taken with respe ct to X).To compu te Var(X|Y ), first expressVar(X|Y = y) = E[(X − E[X|Y = y])2|Y = y]as a function of y.Law of conditional variances:Var(X) = E[Var(X|Y )] + Var(E[X|Y ])(equality betwe en two real numbers)1814 Sum of a random number of iid RVsN discrete RV, Xii.i.d and independent of N.Y = X1+ . . . + XN. Then:E[Y ] = E[X]E[N]Var(Y ) = E[N]Var(X) + (E[X])2Var(N)1915 Covariance and CorrelationCov(X, Y ) = E[(X − E[X])(Y − E[Y ])]= E[XY ] − E[X]E[Y ]If th e covariance is positive, the random variables vary togeth er; ifit is negative, the y vary inve rse l y.Correlation: (has no dimension)ρ =Cov(X, Y )σXσY∈ [−1, 1]By d efinition, X, Y are u ncorrelated if and only if Cov(X, Y ) = 0.Remark: X, Y independent ⇒ Cov(X, Y ) = 0(the converse is not true)2016 Uniform continuous RV over [a, b]fX(x) =1b−aif a ≤ x ≤ b0 otherwiseFX(x) =0 if x ≤ ax−ab−aif a ≤ x ≤ b1 otherwise (x > b)E[X] =a + b2Var(X) =(b − a)2122117 Exponential RV with parameter λfX(x) =λe−λxif x ≥ 00 otherwiseFX(x) =1 − e−λxif x ≥ 00 otherwiseE[X] =1λVar(X) =1λ22218 Normal RV with parameters (µ, σ2)fX(x) =1σ√2πe−(x−µ)2/2σ2E[X] = µVar(X) = σ223Limit Theorems (1)• Markov Inequality If X is a nonnegative random variablewith mean µ, thenP(X ≥ a) ≤µa.• Chebyshev Inequality If X is a random variable with meanµ and variance σ2, th enP(| X − µ |≥ c) ≤σ2c2, for positive c.• Convergence in Probability Let Y1, Y2, .. . be a sequence ofrandom variables (not necessarily independent), and let a be areal number. We say that the s equence Ynconverges to a inprobability, if for every ǫ > 0, we have P(|Yn− a| ≥ ǫ) → 0.Intuitively, as n grows large, the random variable Yntakes onthe value a with probabi l ity 1.24Limit Theorems (2)• Central Limit Theorem: Let X1, X2, .. . be a sequence of i.i. d.random variables with mean µ and variance σ2. We define thenormalized sum Zn=X1+X2+...+Xn−nµσ√nThe CDF of Znconverges to the standard normal CDFpointwise, i . e. P(Zn≤ z) → Φ(z) for eve ry z.• Normal approximation based on CLT: If you have the sum of ni.i.d. random variables, to compute its CDF (and hence someprobability), first subtract the mean and di v i de by the standarddeviation and treat what you get as a standard normal randomvariable. This approximation gets better with
View Full Document
Unlocking...