1 The basics of probability theory E R Morey BasicsOfProbabilityTheory tex and pdf September 8 2010 1 1 1 1 1 What does probability mean What is the probability of some event In the street sense of the word probability of an event is a measure of likeliness how likely is it that the event will happen the more likely it is that the event will occur the higher the probability number That is if the likelihood of an event occurring increases its probability increases Note that probability is from probable The guy on the street would likely put a few restrictions on this measure of likelihood He would probably add 1 The probability of an event cannot be less than zero 2 The probability of an event cannot be greater than 1 greater than 100 3 The probability of something happening is 1 4 And the probability of nothing happening is zero Us statisticans agree with the guy on the street that probability has these four properties But these are just conventions For example there is no reason likelihood could not be measured on a scale from 3 to 1 5 where 3 corresponds to certainty On Mars they might have a di erent range on probability Note a few things about this street de nition of probability It is an exante concept not an expost concept once something has happened we know what happened what event occurred Once an event has occured it is certain it is not a random variable and does not have a probability of occurance 1 Sampling and probability are two side of the same thing Consider the question of whether it will rain in Boulder tomorrow as a function of what the weather in Boulder is today and the weather elsewhere Let s assume that the weather tomorrow is a function of the both of these observables plus some random component A sample is a realization of an experiment with uncertain outcomes the realization of a stochastic data generating process That is tomorrow s weather is today a RV with some probability density function where that probability density function depends on the weather today both in Boulder and elsewhere Tomorrow s weather will be a draw from that distribution Tomorrow we might draw sample a sunny day or we might draw sample a rainy day Today all we can ask is what is the probability that tomorrow will be sunny given today is bla bla bla For example consider a discrete distribution with three spikes one for the probability of rain one for sun and one for snow where the height of each spike depends on whether today was rain sun or snow possiblly insert graph here maybe create in excel and use the excel macro that converts stu to tex What if the issue was tomorrow s temperature Imagine we assumed temperature tomorrow would be a random draw from a normal distribution whose mean was today s temperature or whose mean was a weighted average of today and yesterday s temperatures An observation what happens on a day is a draw from the population of possible weathers Or said di erently there is a underlying process with a random component that will generate tomorrow s weather and tomorrow s actual weather is one of many possible outcomes Tomorrow we will sample the weather we are forced to sample it and see what we get We want to determine or estimate the probability that an observation in a sample will be event A For example the probability that it will rain tomorrow rain is an event The underlying process and what happens tomorrow can all be thought of in terms of randomly drawing a colored ball from an urn For example based on today s weather the urn for tomorrow s weather holds two white balls white means snow ve red balls red means sunny and three grey balls grey means 2 rain 1 Tomorrow s weather is a draw from the urn It will be either snow rain or sun 2 Once the ball is drawn the outcome is known Beforehand we want to know the likelihood probability of each outcome For example in my research I am often interested in the probability that individual i will choose alternative j from some nite number of alternatives J The probably of choosing each alternative will depend on its characteristics and the characteristics of the other alternatives in the choice set Guys in marketing and transportation love these models This type of model is called a discrete choice model I am going skiing which area will I choose Every day unmarried people decide whether to get married or not every day married people decide whether to get divorced Coke or Pepsi McDonalds Wendys or Burger King In terms of urns every possible con guration of characteristics for the J alternatives is represented by a di erent urn they could be hundred or thousand of di erent con gurations of alternatives so that number of urns Each of the di erent urns will contain numbered balls the number on the ball corresponding to the number of the alternative Di erent urns will have di erent proportions of the di erent numbered balls An observed choice is a draw from the urn that represents the current con guration of the alternatives in terms of their characteristics For example if the problem is determining the probability of where to ski as a function of the number of ski areas their locations and their characteristics one could imagine an urn for each possible con guation of ski areas To simplify assume there are always 5 ski areas and their locations are xed Assume that what varys from day to day and across ski areas is snow conditions good or bad and temperature cold nice and too warm How many urns would be needed 1 If today s weather were di erent the composition of the balls in this urn would be di erent or thinking of it another way there could be a di erent urn for every possible state of today s weather In such a world the weather predictor might own three urns the urn for if it rained today the urn for if it snowed today and the sun urn be able to observe today s weather and at midnight predict tomorrow s weather by drawing a ball from the appropriate urn every morning The draw is the weather forcast for the next day Note that the weather person s draw will not necesarily be tomorrow s weather That is determined by a draw by the weather god 2 Note the assumption that only three things can happen snow rain or sun so weather is assumed a discretely distributed random variable It can take only three values 3 In each urn there would be balls of ve colors a di erent color for each ski area The proportion of balls by color would vary across urns One wakes up in the morning nds the urn that corresponds to today s