1 Probability Density Functions (PDF) For a continuous RV X with PDF fX (x), b 6.041/6.431 Probabilistic Systems P (a ≤ X ≤ b)= fX (x)dx a Analysis P (X ∈ A)= fX (x)dx A Quiz II Review Properties: Fall 2010 • Nonnegativity: fX (x) ≥ 0 ∀x • Normalization: ∞ fX (x)dx =1 −∞ 1 2 3 Mean and variance of a continuous RV 2 PDF Interpretation ∞ E[X]= xfX (x)dx Caution: fX (x) = x) = P (X −∞ 2Var(X)= E (X − E[X])• if X is continuous, P (X = x)=0 ∀x!! ∞ • fX (x)can be ≥ 1 = (x − E[X])2fX (x)dx −∞ Interpretation: “probability per unit length” for “small” lengths = E[X2] − (E[X])2 (≥ 0) around x ∞ E[g(X)] = g(x)fX (x)dx −∞ P (x ≤ X ≤ x + δ) ≈ fX (x)δE[aX + b]= aE[X]+ b Var(aX + b)= a 2Var(X) 3 4 4 Cumulative Distribution Functions 5 Uniform Random Variable Definition: If X is a uniform random variable over the interval [a,b]: FX (x)= P (X ≤ x) ⎧ monotonically increasing from 0 (at −∞)to 1 (at +∞). ⎨ 1 if a ≤ x ≤ b • Continuous RV (CDF is continuous in x): fX (x)= b−a ⎩ 0 otherwise x ⎧FX (x)= P (X ≤ x)= fX (t)dt ⎪ ⎪ 0 if x ≤ a−∞ ⎨ FX (x)= x−a if a ≤ x ≤ bdFXb−a ⎪fX (x)= (x) ⎪ ⎩dx 1 otherwise (x>b) • Discrete RV (CDF is piecewise constant): b − a E[X]= FX (x)= P (X ≤ x)= pX (k)2 k≤x (b − a)2 var(X)= pX (k)= FX (k) − FX (k − 1) 12 5 6 6 Exponential Random Variable X is an exponential random variable with parameter λ: 7 Normal/Gaussian Random Variables ⎧ ⎨ λe−λx if x ≥ 0 General normal RV: N(μ, σ2): fX (x)= ⎩ 0 otherwise fX (x)= √ 1 e −(x−μ)2/2σ2 ⎧ σ 2π ⎨ −λx1 − e if x ≥ 0 E[X]= μ, Var(X)= σ2 FX (x)= ⎩ 0 otherwise Property: If X ∼ N (μ, σ2)and Y = aX + b 1 1 E[X]= λ var(X)= λ2 then Y ∼ N (aμ + b, a2σ2) Memoryless Property: Given that X>t, X − t is an exponential RV with parameter λ 7 88Normal CDF Standard Normal RV: N(0, 1) CDF of standard normal RV Y at y: Φ(y) - given in tables for y ≥ 0 -for y< 0, use the result: Φ(y)=1 − Φ(−y) To evaluate CDF of a general standard normal, express it as a function of a standard normal: X ∼ N(μ, σ2) ⇔ X − μ σ ∼ N (0, 1) P (X ≤ x)= P X − μ σ ≤ x − μ σ =Φ x − μ σ 9 9Joint PDF Joint PDF of two continuous RV X and Y : fX,Y (x, y) P (A)= A fX,Y (x, y)dxdy Marginal pdf: fX (x)= ∞ −∞ fX,Y (x, y)dy E[g(X, Y )] = ∞ −∞ ∞ −∞ g(x, y)fX,Y (x, y)dxdy Joint CDF: FX,Y (x, y)= P (X ≤ x, Y ≤ y) 10 10 Independence By definition, X, Y independent ⇔ fX,Y (x, y)= fX (x)fY (y) ∀(x, y) If X and Y are independent: • E[XY ]=E[X]E[Y ] • g(X)and h(Y ) are independent • E[g(X)h(Y )] = E[g(X)]E[h(Y )] 11 11 Conditioning on an event Let X be a continuous RV and A be an event with P (A) > 0, fX|A(x)= ⎧ ⎨ ⎩ fX (x) P (X∈A) if x ∈ A 0 otherwise P (X ∈ B|X ∈ A)= B fX|A(x)dx E[X|A]= ∞ −∞ xfX|A(x)dx E[g(X)|A]= ∞ −∞ g(x)fX|A(x)dx 12 12 Conditioning on a RV X, Y continuous RV If A1,...,An are disjoint events that form a partition of the sample space, fX,Y (x, y)fX|Y (x|y)= fY (y) n ∞fX (x)= P (Ai)fX|Ai (x)(≈ total probability theorem) fX (x)= fY (y)fX|Y (x|y)dy (≈ totalprobthm)i=1 −∞ n E[X]= P (Ai)E[X|Ai] (total expectation theorem) Conditional Expectation: i=1 ∞ n E[X|Y = y]= xfX|Y (x|y)dx E[g(X)] = P (Ai)E[g(X)|Ai] −∞ ∞ i=1 E[g(X)|Y = y]= g(X)fX|Y (x|y)dx −∞ ∞ E[g(X, Y )|Y = y]= g(x, y)fX|Y (x|y)dx −∞ 13 14 13 Continuous Bayes’ Rule Total Expectation Theorem: ∞ X, Y continuous RV, N discrete RV, A an event. E[X]= E[X|Y = y]fY (y)dy ∞∞ −∞E[g(X)] = −∞ E[g(X)|Y = y]fY (y)dy fX|Y (x|y)= fY |X (fyY |x(y)f) X (x)= fY f|YX |(X y(|xy|)tf)X fX (x() t)dt −∞ P (A)fY |A(y) P (A)fY |A(y) ∞ E[g(X, Y )] = E[g(X, Y )|Y = y]fY (y)dy P (A|Y = y)= fY (y)= fY |A(y)P (A)+ fY |Ac (y)P (Ac) −∞ P (N = n|Y = y)= pN (n)fY |N (y|n)= pN (n)fY |N (y|n) fY (y) i pN (i)fY |N (y|i) 15 16 14 Derived distributions 15 Convolution Def: PDF of a function of a RV X with known PDF: Y = g(X). W = X + Y ,with X, Y independent. Method: • Discrete case: • Get the CDF: pW (w)= pX (x)pY (w − x) FY (y)= P (Y ≤ y)= P (g(X) ≤ y)= fX (x)dx x x|g(x)≤y • Continuous case: • Differentiate: fY (y)= dFY (y) ∞ dy fW (w)= fX (x)fY (w − x) dx Special case:if Y = g(X)= aX + b, fY (y)= 1 fX ( x−b ) −∞ |a| a 17 18 16 Law of iterated expectations Graphical Method: • put the PMFs (or PDFs) on top of each other E[X|Y = y]= f(y)is a number. • flip the PMF (or PDF) of YE[X|Y ]= f (Y ) is a random variable (the expectation is taken with respect to X). • shift the flipped PMF (or PDF) of Y by w To compute E[X|Y ], first express E[X|Y = y] as a function of y. • cross-multiply and add (or evaluate the integral) Law of iterated expectations: In particular, if X, Y are independent and normal, then E[X]= E[E[X|Y ]] W = X + Y is normal. (equality between two real numbers) 19 2017 Law of Total Variance Var(X|Y ) is a random variable that is a function of Y 18 Sum of a random number of iid RVs (the variance is taken with respect to X). To compute Var(X|Y ), first express N discrete RV, Xi i.i.d and independent of N. Var(X|Y = y)= E[(X − E[X|Y = y])2|Y = y] Y = X1 + ... + XN . Then: as a function of y. E[Y ]= E[X]E[N] Var(Y )= E[N ]Var(X)+(E[X])2Var(N) Law of conditional variances: Var(X)= E[Var(X|Y )] + Var(E[X|Y ]) (equality between two real numbers) 21 22 19 Covariance and Correlation Cov(X, Y )= E[(X − E[X])(Y − E[Y ])] Correlation Coefficient: (dimensionless) = E[XY ] − E[X]E[Y ] Cov(X, Y ) • By definition, X, Y are uncorrelated ⇔ Cov(X, Y )= 0. ρ = ∈ [−1, 1]σX σY • If X, Y independent ⇒ X and Y are uncorrelated. (the ρ =0 ⇔ X and Y are uncorrelated. converse is not true) |ρ| =1 …
View Full Document
Unlocking...