Probability and Statistics!Robert Stengel! Robotics and Intelligent Systems MAE 345 !Princeton University, 2013"• Concepts and reality"– Interpretations of probability"– Measures of probability"• Scalar uniform and Gaussian distributions"• Hypothesis testing"• Bayess Law"• Bayesian Belief Networks"• Propagation of the states probability distribution"Copyright 2013 by Robert Stengel. All rights reserved. For educational use only.!http://www.princeton.edu/~stengel/MAE345.html!Learning Objectives!Probability"• ... a way of expressing knowledge or belief that an event will occur or has occurred"Statistics"• The science of making effective use of numerical data relating to groups of individuals or experiments"How Do Probability and Statistics Relate to Robotics and Intelligent Systems?"• Decision-making under uncertainty"• Controlling random dynamic processes" Concepts and Reality "(Papoulis)"• Theory may be exact"– Deals with averages of phenomena with many possible outcomes"– Based on models of behavior"• Application can be only approximate"– Measure of our state of knowledge or belief that something may or may not be true"– Subjective assessment"A : eventP(A) : probability of eventnA: number of times A occurs experimentallyN :total number of trialsP(A) ≈nANInterpretations of Probability (Papoulis)"• Axiomatic Definition (Theoretical interpretation)"– Probability space, abstract objects (outcomes), and sets (events)"– Axiom 1: Pr(Ai) ≥0"– Axiom 2: Pr( certain event) = 1 = Pr [all events in probability space (or universe)]"– Axiom 3: With no common elements, "• Relative Frequency (Empirical interpretation)"Pr Ai∪ Aj( )= Pr Ai( )+ Pr Aj( )Pr Ai( )= limN →∞nAiN⎛⎝⎜⎞⎠⎟N = number of trials (total)"nAi = number of trials with attribute Ai"Interpretations of Probability (Papoulis)"• Classical ( Favorable outcomes interpretation)"• Measure of belief (Subjective interpretation)"– Pr(Ai) = measure of belief that Ai is true (similar to fuzzy sets)"– Informal induction precedes deduction"– Principle of insufficient reason (i.e., total prior ignorance):"• e.g., if there are 5 event sets, Ai, i = 1 to 5, Pr(Ai) = 1/5 = 0.2"Pr Ai( )=nAiNN is finite"nAi = number of outcomes favorable toAi!Favorable Outcomes Example: Probability of Rolling a 7 with Two Dice "(Papoulis)"• Proposition 1: 11 possible sums, one of which is 7"Pr Ai( )=nAiN=111• Proposition 3: 36 possible outcomes, distinguishing between the two dice"– 6 pairs: 1-6, 2-5, 3-4, 6-1, 5-2, 4-3"• Proposition 2: 21 possible pairs, not distinguishing between dice"– 3 pairs: 1-6, 2-5, 3-4"Pr Ai( )=nAiN=321Pr Ai( )=nAiN=636Propositions are knowable and precise; outcome of rolling the dice is not." Steps in a Probabilistic Investigation "(Papoulis)"1) Physical (Observation): Determine probabilities, Pr(Ai), of various events, Ai, by experiment"• Experiments cannot be exact"2) Conceptual (Induction): Assume that Pr(Ai) satisfies certain axioms and theorems, allowing deductions about other events, Bi, based on Pr(Bi)"• Build a model"3) Physical (Deduction): Make predictions of Bi based on Pr(Bi)"Empirical (or Relative) Frequency of Discrete, Mutually Exclusive Events in Sample Space"• N = total number of events"• ni = number of events with value xi"• I = number of different values"• xi = ordered set of hypotheses or values"Pr xi( )=niNin [0,1]; i = 1 to Ix is a random variable"Empirical (or Relative) Frequency of Discrete, Mutually Exclusive Events in Sample Space"• Equivalent sets"Ai= x ∈U x = xi{ }; i = 1 to I• Cumulative probability over all sets"Pr Ai( )i =1I∑= Pr xi( )i =1I∑=1Nnii =1I∑= 100.050.10.150.20.250.31 2 3 4 5Pr(x)• x is a random variable"Cumulative Probability, Pr(x ≥/≤ a), and Discrete Measurements of a Continuous Variable"Suppose x represents a continuum of colors"xi is the center of a band in x!00.10.20.30.40.50.60.70.80.911 2 3 4 5Pr(x)Cum Pr(x) ≥ aCum Pr(x) ≤ aPr xi± Δx / 2( )= ni/ NPr xi± Δx / 2( )= 1i =1I∑Probability Density Function, pr(x)"Cumulative Distribution Function, Pr(x <X)"Probability density function!Pr x < X( )= pr x( )dx−∞X∫Cumulative distribution function!pr xi( )=Pr xi± Δx / 2( )ΔxPr xi± Δx / 2( )= pr xi( )Δxi =1I∑Δx→0I →∞⎯ →⎯⎯ pr x( )dx−∞∞∫= 1i =1I∑Probability Density Function, pr(x) Cumulative Distribution Function, Pr(x <X)"Pr x < X( )= pr x( )dx−∞X∫Random Number Example"Statistical -- not deterministic -- properties prior to actual event!00.10.20.30.40.50.60.70.80.91RandomRandomDeterministic00.10.20.30.40.50.60.70.80.9100.10.20.30.40.50.60.70.80.910.000.100.200.300.400.500.600.700.800.900.18 0.54 0.49 0.49 0.02 0.73 0.88"0.81 0.46 0.84 0.16 0.89 0.30 0.03"0.10 0.20 0.30 0.40 0.50 0.60 0.70"=RAND() =RAND() =RAND() =RAND() =RAND() =RAND() =RAND()=RAND() =RAND() =RAND() =RAND() =RAND() =RAND() =RAND()0.1 0.2 0.3 0.4 0.5 0.6 0.7• Excel spreadsheet: 2 random rows and one deterministic row ""– [RAND()] generates a uniform random number on each call!1st Trial"2nd Trial" 3rd Trial"4th Trial"Output for 4th trial!Once the experiment is over, the results are determined "Properties of Random Variables"• Mode"– Value of x for which pr(x) is maximum"x = E(x) = x pr x( )dx−∞∞∫• Median"– Value of x corresponding to 50th percentile"– Pr(x < median) = Pr(x > median) = 0.5"• Mean"– Value of x corresponding to statistical average"• First moment of x = Expected value of x" Moment arm" Force"Expected Values"• Second central moment of x = Variance"– Variance from the mean value rather than from zero"– Smaller value indicates less uncertainty in the value of x"E x − x( )2⎡⎣⎤⎦=σx2= x − x( )2pr x( )dx−∞∞∫• Expected value of a function of x"E f (x)[ ]= f (x) pr x( )dx−∞∞∫x = E(x) = x pr x( )dx−∞∞∫• Mean Value is the first moment of x"Expected Value is a Linear Operation"E x1+ x2[ ]= x1+ x2( )pr x( )dx−∞∞∫= x1pr x( )dx−∞∞∫+ x2pr x( )dx−∞∞∫= E x1[ ]+ E x2[ ]E k x[ ]= k x pr x( )dx−∞∞∫= k x pr x( )dx−∞∞∫= k E x[ ]Expected value of sum of random variables"Expected value of constant times random