Probability1Probability•Random variables•Atomic events•Sample space2Probability•Events•Combining events3Probability•Measure:•disjoint union:•e.g.:•interpretation:•Distribution:•interpretation:•e.g.:4ExampleWeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.145Bigger exampleWeatherAAPL priceupsamedownsunrain0.030.050.020.070.120.05Weatherupsamedownsunrain0.140.230.090.060.100.04LAX PIT6Notation•X=x: event that r.v. X is realized as value x•P(X=x) means probability of event X=x•if clear from context, may omit “X=”•instead of P(Weather=rain), just P(rain)•complex events too: e.g., P(X=x, Y!y)•P(X) means a function: x ! P(X=x)7Functions of RVs•Extend definition: any deterministic function of RVs is also an RV•E.g., WeatherAAPL priceupsamedownsunrain3830508Sample v. population•Suppose we watch for 100 days and count up our observationsWeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.14WeatherAAPL priceupsamedownsunrain9Law of large numbers•If we take a sample of size N from distribution P, count up frequencies of atomic events, and normalize (divide by N) to get a distribution P•Then P ! P as N ! "~~10Working w/ distributions•Marginals•Joint11MarginalsWeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.1412MarginalsWeatherAAPL priceupsamedownsunrain0.030.050.020.070.120.05Weatherupsamedownsunrain0.140.230.090.060.100.04LAX PIT13Law of total probability•Two RVs, X and Y•Y has values y1, y2, …, yk•P(X) = 14Working w/ distributions•Conditional:•Observation•Consistency•Renormalization•Notation:WeatherCoinHTsunrain0.150.150.350.3515Conditionals in the literatureWhen you have eliminated the impossible, whatever remains, however improbable, must be the truth.—Sir Arthur Conan Doyle, as Sherlock Holmes16ConditionalsWeatherAAPL priceupsamedownsunrain0.030.050.020.070.120.05Weatherupsamedownsunrain0.140.230.090.060.100.04LAX PIT17In general•Zero out all but some slice of high-D table•or an irregular set of entries•Throw away zeros•unless irregular structure makes it inconvenient•Renormalize•normalizer for P(. | event) is P(event)18Conditionals•Thought experiment: what happens if we condition on an event of zero probability?19•P(X | Y) is a function: x, y ! P(X=x | Y=y)•As is standard, expressions are evaluated separately for each realization:•P(X | Y) P(Y) means the function x, y ! Notation20Exercise21•X and Y are independent if, for all possible values of y, P(X) = P(X | Y=y)•equivalently, for all possible values of x, P(Y) = P(Y | X=x)•equivalently, P(X, Y) = P(X) P(Y)•Knowing X or Y gives us no information about the otherIndependence22WeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.140.30.70.3 0.5 0.2Independence: probability = product of marginals23Expectations•How much should we expect to earn from our AAPL stock?Weatherupsamedownsunrain+10-1+10-1WeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.1424Linearity of expectation•Expectation is a linear function of numbers in bottom table•E.g., change -1s to 0s or to -2sWeatherupsamedownsunrain+10-1+10-1WeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.1425Conditional expectation•What if we know it’s sunny?Weatherupsamedownsunrain+10-1+10-1WeatherAAPL priceupsamedownsunrain0.090.150.060.210.350.1426Independence and expectation•If X and Y are independent, then:•Proof:27Variance•Two stocks: one as above, other always earns 0.1 each day•Same expectation, but one is much more variable•Measure of variability: variance28Variance•If zero-mean: variance = E(X2)•Ex: constant 0 v. coin-flip ±1•In general: E((X – E(X))2)29Exercise: simplify the expression for variance•E((X – E(X))2)30Covariance•Suppose we want an approximate numeric measure of (in)dependence•Consider the r.v. XY•if X, Y are typically both +ve or both -ve•if X, Y are independent31Covariance•cov(X, Y) = •Is this a good measure of dependence?•Suppose we scale X by 10:32Correlation•Like covariance, but control for variance of individual r.v.s•cor(X, Y) =33Correlation v. independence•Equal probability on each point•Are X and Y independent?•Are X and Y uncorrelated?XY!!"!!#!!!$"$!#34Correlation v. independence•Equal probability on each point•Are X and Y independent?•Are X and Y uncorrelated?!!"!!#!!!$"$!#XY35Law of large numbers•Sample mean = expectation calculated from a sample = •More general form of law:•If we take a sample of size N from distribution P with mean " and compute sample mean "•Then " ! " as N ! "~~36CLT•Central limit theorem: for a sample of size N, population mean ", population variance #2, the sample average has•mean•variance37CLT proof•Assume mu = 0 for simplicity38•For any X, Y, C•P(X | Y, C) P(Y | C) = P(Y | X, C) P(X | C)•Simple version (without context)•P(X | Y) P(Y) = P(Y | X) P(X)•Can be taken as definition of conditioningBayes RuleRev. Thomas Bayes1702–176139Bayes rule: usual form•Take symmetric form •P(X | Y) P(Y) = P(Y | X) P(X)•Divide by P(Y)40Exercise•You are tested for a rare disease, emacsitis—prevalence 3 in 100,000•Your receive a test that is 99% sensitive and 99% specific•sensitivity = P(yes | emacsitis)•specificity = P(no | ~emacsitis)•The test comes out positive•Do you have
View Full Document
Unlocking...