Review of ProbabilityRandom VariableRandom VariableTwo Types of Random VariablesPDF for Continuous RV Most Commonly Used PDF: GaussianZero-Mean Gaussian PDFEffect of Variance on Gaussian PDFWhy Is Gaussian Used?Joint PDF of RVs X and YConditional PDF of Two RVsSlide Number 12Independent RV’sIndependent and Dependent Gaussian PDFsAn “Independent RV” ResultCharacterizing RVsMotivating Idea of Mean of RVTheoretical View of MeanAside: Probability vs. StatisticsVariance of RVMotivating Idea of CorrelationIllustrating 3 Main Types of CorrelationProb. Theory View of CorrelationSlide Number 24Independence vs. UncorrelatedConfusing Covariance and Correlation TerminologyCovariance and Correlation For Random Vectors…A Few Properties of Expected ValueJoint PDF for GaussianLinear Transform of Jointly Gaussian RVsMoments of Gaussian RVsChi-Squared Distribution1Review of Probability2Random Variable DefinitionNumerical characterization of outcome of a random event Examples1) Number on rolled dice2) Temperature at specified time of day3) Stock Market at close4) Height of wheel going over a rocky road3Random Variable Non-examples1) ‘Heads’ or ‘Tails’ on coin2) Red or Black ball from urnBut we can make these into RV’s Basic Idea – don’t know how to completely determine what value will occur– Can only specify probabilities of RV values occurring.4Two Types of Random VariablesRandom VariableDiscrete RV• Die• StocksContinuous RV• Temperature• Wheel height5Given Continuous RV X… What is the probability that X = x0?Oddity : P(X = x0) = 0Otherwise the Prob. “Sums” to infinityNeed to think of Prob. Density Function (PDF)pX(x)xThe Probability density function of RV X∆+oxox∫∆+==∆+<<ooxxXdxxpxXxP)(shownarea )(00PDF for Continuous RV6Most Commonly Used PDF: Gaussianm & σare parameters of the Gaussian pdfm = Mean of RV Xσ= Standard Deviation of RV X (Note: σ> 0)σ2= Variance of RV X222/)(21)(σπσmxXexp−−=A RV X with the following PDF is called a Gaussian RVNotation: When X has Gaussian PDF we say X ~ N(m,σ2)7 Generally: take the noise to be Zero Mean22221)(σπσxxexp =Zero-Mean Gaussian PDF8pX(x)xArea within ±1 σof mean = 0.683 = 68.3%σ σx = mSmall σSmall Variability (Small Uncertainty)pX(x)xLarge σLarge Variability (Large Uncertainty)pX(x)xEffect of Variance on Gaussian PDF9Central Limit theorem (CLT)The sum of N independent RVs has a pdfthat tends to be Gaussian as N → ∞So What! Here is what : Electronic systems generate internal noise due to random motion of electrons in electronic components. The noise is the result of summing the random effects of lots of electrons.CLT appliesGuassian NoiseWhy Is Gaussian Used?10Describes probabilities of joint events concerning X and Y. For example, the probability that X lies in interval [a,b] and Y lies in interval [a,b] is given by:),(yxpXY{ }∫∫=<<<<badcXYdxdyyxpdYcbXa ),()( and )(PrThis graph shows a Joint PDFGraph from B. P. Lathi’s book: Modern Digital & Analog Communication SystemsJoint PDF of RVs X and Y11When you have two RVs… often ask: What is the PDF of Y if X is constrained to take on a specific value. In other words: What is the PDF of Y conditioned on the fact X is constrained to take on a specific value. Ex.: Husband’s salary X conditioned on wife’s salary = $100K? First find all wives who make EXACTLY $100K… how are their husband’s salaries distributed. Depends on the joint PDF because there are two RVs… but it should only depend on the slice of the joint PDF at Y=$100K. Now… we have to adjust this to account for the fact that the joint PDF (even its slice) reflects how likely it is that Y=$100K will occur (e.g., if Y=105is unlikely then pXY(x,105) will be small); so… if we divide by pY(105) we adjust for this. Conditional PDF of Two RVs12Conditional PDF (cont.)Thus, the conditional PDFs are defined as (“slice and normalize”):≠=otherwise,00)(,)(),()|(|xpxpyxpxypXXXYXY≠=otherwise,00)(,)(),()|(|ypypyxpyxpYYXYYXy is held fixedThis graph shows a Conditional PDFGraph from B. P. Lathi’s book: Modern Digital & Analog Communication Systems“slice and normalize”y is held fixedx is held fixed13Independence should be thought of as saying that: Neither RV impacts the other statistically – thus, the values that one will likely take should be irrelevant to the value that the other hastaken.In other words: conditioning doesn’t change the PDF!!!)()(),()|()()(),()|(||xpypyxpyxpypxpyxpxypXYXYyYXYXXYxXY======Independent RV’s14Independent and Dependent Gaussian PDFsIndependent(zero mean)yxContours of pXY(x,y). If X & Y are independent, then the contour ellipses are alignedwith either the x or y axisDependentyxDifferent slices give different normalized curves Independent(non-zero mean)yxDifferent slices give same normalized curves15RV’s X & Y are independent if:)()(),( ypxpyxpYXXY=)()()()()(),()|(|ypxpypxpxpyxpxypYXYXXXYxXY====Here’s why:An “Independent RV” Result16Characterizing RVs PDF tells everything about an RV– but sometimes they are “more than we need/know” So… we make due with a few Characteristics– Mean of an RV (Describes the centroid of PDF)– Variance of an RV (Describes the spread of PDF)– Correlation of RVs (Describes “tilt” of joint PDF)Mean = Average = Expected ValueSymbolically: E{X}17Motivation First w/ “Data Analysis View”Consider RV X = Score on a test Data: x1, x2,… xNPossible values of RV X : V0V1V2... V1000 1 2 … 100Ni = # of scores of value ViN =(Total # of scores)∑=niiN1Test Average≈ P(X = Vi)∑∑===++===100010011001...iiinNiiNNVNVNVNVNNxxThis is called Data Analysis ViewBut it motivates the Data Modeling ViewProbabilityStatisticsMotivating Idea of Mean of RV18Theoretical View of MeanData Analysis View leads to Probability Theory: This Motivates form for Continuous RV:Probability Density Function∫∞∞−= dxxpxXEX)(}{ For Discrete random Variables :∑==nniXixPxXE1)(}{Probability FunctionData ModelingNotation:XXE =}{Shorthand Notation19Aside: Probability vs. StatisticsProbability Theory» Given a PDF Model» Describe how the data will likely behaveStatistics» Given a set of Data» Determine how the data did behaveThere is no DATA here!!!The PDF models how the data will likely behaveThere is no PDF here!!!The Statistic measures how the data did behave∫∞∞−= dxxpxXEX)(}{Dummy