ECE270: Handout 14Pairs of RVs (Part IV)Outline:1. conditional probability of Y given X when:1) X and Y are both discrete,2) X is discrete and Y is continuous,3) X and Y are both continuous,2. conditional expectation.• Many RVs of practical interest are not independent: the output Y of a communication channelmust depend on the input X in order to convey information. We are interested in computingthe probability of events concerning the RV Y given that we know X = x. We are alsointerested in the expected value of Y given X = x, i.e., E[Y |X = x].• Conditional probability and conditional expectation are very useful tools in solving problems,even in situations where we are only interested in one of the RVs.F Conditional Probability• We consider three cases: 1) when X and Y are both discrete RVs, 2) when X is a discreteand Y is a continuous RV, 3) when X and Y are both continuous RVs.F Conditional Probability When X and Y are Both Discrete• The conditional pmf of Y given X = xjispY |X(yk|xj) = P (Y = yk|X = xj) =P (Y = yk, X = xj)P (X = xj)=pX,Y(xj, yk)pX(xj)• The ab ove equation implies that the joint pmf of X and Y can be written as the product ofa conditional pmf and a marginal pmf:pX,Y(xj, yk) = pY |X(yk|xj)pX(xj)• Using this, we can find the marginal pmf of Y in terms of the conditional pmf pY |X(yk|xj):pY(yk) =Xall xjpX,Y(xj, yk) =Xall xjpY |X(yk|xj)pX(xj)• If X and Y are independent, then:pY |X(yk|xj) = P (Y = yk|X = xj)=P (Y = yk, X = xj)P (X = xj)=P (Y = yk)P (X = xj)P (X = xj)= P (Y = yk) = pY(yk) ∀ yk, xjThe knowledge that X = xjdoes not affect the probability of events involving Y .EXAMPLE 1Suppose the total number of defects X on a chip is a Poisson RV with mean α, i.e., pX(n) =P (X = n) =αnn!e−α. Each defect has a probability p of falling in a specific region R, and thelocation of each defect is independent of the locations of other defects. Let Y be the numberof defects that fall in the region R. What is the pmf of Y ?F Conditional Probability when X is Discrete and Y is Continuous• The conditional CDF of Y given X = xjisFY |X(y|xj) = P (Y ≤ y|X = xj) =P (Y ≤ y, X = xj)P (X = xj)• The conditional pdf of Y given X = xjisfY |X(y|xj) =ddyFY |X(y|xj) =dP (Y ≤ y, X = xj)/dyP (X = xj)• If X and Y are independent, then:FY |X(y|xj) = P (Y ≤ y|X = xj) =P (Y ≤ y, X = xj)P (X = xj)=P (Y ≤ y)P (X = xj)P (X = xj)= FY(y)and consequently:fY |X(y|xj) = fY(y)The knowledge that X = xjdoes not affect the probability of events involving Y .EXAMPLE 2The input X to a communication channel assumes the values +1 or −1 with probabilities1/3 and 2/3, respectively. The output of the channel Y is given by Y = X + N where N isa zero mean unit variance Gaussian RV. The input X and additive noise N are independent.a) Find FY |X(y| + 1) = P (Y ≤ y|X = +1) and FY |X(y| − 1) = P (Y ≤ y|X = −1).Page 2b) Find P (Y > 0|X = +1) and P (Y > 0|X = −1).c) Find P (Y > 0).d) Find P (X = +1|Y > 0) and P (X = −1|Y > 0). Given that Y > 0 which one ismore likely, X = +1 or X = −1?F Conditional Probability When X and Y are Both Continuous• If X is a continuous RV then P (X = x) is zero all for x values. Hence, the conditional CDFof Y given X = x is defined through a limiting procedure:FY |X(y|x) = limh→0P (Y ≤ y, x < X ≤ x + h)P (x < X ≤ x + h)= limh→0Ry−∞Rx+hxfX,Y(s, t)dsdtRx+hxfX(s)ds=Ry−∞fX,Y(x, t)hdtfX(x)h=Ry−∞fX,Y(x, t)dtfX(x)• The conditional pdf of Y given X = x is:fY |X(y|x) =ddyFY |X(y|x) =fX,Y(x, y)fX(x)• The above equation implies that the joint pdf of X and Y can be written as the product ofa conditional pdf and a marginal pdf:fX,Y(x, y) = fY |X(y|x)fX(x)• Using this, we can find the marginal pdf of Y in terms of the conditional p df pY |X(y|x):fY(y) =Z∞−∞fX,Y(x, y)dx =Z∞−∞fY |X(y|x)fX(x)dxandFY(y) =Zy−∞fY(t)dt =Zy−∞µZ∞−∞fX,Y(x, t)dx¶dt=Z∞−∞Zy−∞fX,Y(x, t)dtdx =Z∞−∞FY |X(y|x)fX(x)dx• If X and Y are independent, then:FY |X(y|x) = FY(y) fY |X(y|x) = fY(y)Page 3EXAMPLE 3X is selected at random from the unit interval; Y is then selected at random from the interval(0, X ). Find the CDF and pdf of Y .F Conditional Expectation• We are interested in finding the expected value of Y given X = x, i.e., E[Y |X = x] = E[Y |x].The conditional expectation E[Y |x] can be viewed a function of x, i.e., g(x) = E[Y |x].So it makes sense to talk about the random variable g(X) = E[Y |X] and its expectationE[g(X )] = E[E[Y |X ]]. we can show that:E[Y ] = E[E[Y |X]]Proof: we recognize three cases: case i) X and Y are both discrete, case ii) If X is discreteand Y is continuous, case iii) If X and Y are both continuous.• case i) If X and Y are both discrete:E[E[Y |X]] =Xall xjE[Y |xj]P (X = xj)=Xall xjXall ykykpY |X(yk|xj)pX(xj)=Xall xjXall ykykpX,Y(xj, yk) =Xall ykykpY(yk) = E[Y ]• case ii) If X is discrete and Y is continuous:E[E[Y |X]] =XxjE[Y |xj]P (X = xj)=Xxj·Z∞−∞yfY |X(y|xj)dy¸P (X = xj) =Z∞−∞yfY(y)dy = E[Y ]• case iii) If X and Y are both continuous:E[E[Y |X]] =Z∞−∞E[Y |x]fX(x)dx=Z∞−∞·Z∞−∞yfY |X(y|x)dy¸fX(x)dx=Z∞−∞Z∞−∞yfX,Y(x, y)dxdy =Z∞−∞yfY(y)dy = E[Y ]Page 4EXAMPLE 4 (back to example 1)Considering Example 1 find E[Y ] using conditional expectation.EXAMPLE 5 (back to example 2)Considering Example 2 find E[Y ] using conditional expectation.EXAMPLE 6 (back to example 3)Considering Example 3 find E[Y ] using conditional expectation.Page