Page1Chapter1ProbabilitiesandrandomvariablesProbabilitytheoryisasystematicmethodfordescribingrandomnessanduncertainty.Itpre-scribesasetofrulesformanipulatingandcalculatingprobabilitiesandexpectations.Ithasbeenappliedinmanyareas:gambling,insurance,thestudyofexperimentalerror,statisticalinference,andmore.Onestandardapproachtoprobabilitytheory(butnottheonlyapproach)startsfromtheconceptofasamplespace,whichisanexhaustivelistofpossibleoutcomesinanexperimentorother•samplespacesituationwheretheresultisuncertain.Subsetsofthelistarecalledevents.Forexample,inthe•eventsverysimplesituationwhere3coinsaretossed,thesamplespacemightbeS={hhh,hht,hth,htt,thh,tht,tth,ttt}.NoticethatScontainsnothingthatwouldspecifyanoutcomelike“thesecondcoinspun17times,wasintheairfor3.26seconds,rolled23.7incheswhenitlanded,thenendedwithheadsfacingup”.Thereisaneventcorrespondingto“thesecondcoinlandedheads”,namely,{hhh,hht,thh,tht}.Eachelementinthesamplespacecorrespondstoauniquelyspecifiedoutcome.Thechoiceofasamplespace—thedetailwithwhichpossibleoutcomesaredescribed—dependsonthesortofeventswewishtodescribe.Thesamplespaceisconstructedtomakeiteasiertothinkpreciselyaboutevents.Inmanycases,youwillfindthatyoudon’tactuallyneedanexplic-itlydefinedsamplespace;itoftensufficestomanipulateeventsviaasmallnumberofrules(tobespecifiedsoon)withoutexplicitlyidentifyingtheeventswithsubsetsofasamplespace.Iftheoutcomeoftheexperimentcorrespondstoapointofasamplespacebelongingtosomeevent,onesaysthattheeventhasoccurred.Forexample,withtheoutcomehhheachoftheevents{notails},{atleastonehead},{moreheadsthantails}occurs,buttheevent{evennumberofheads}doesnot.Theuncertaintyismodelledbyaprobabilityassignedtoeachevent.Theprobabibilityofan•probabilityeventEisdenotedbyPE.OnepopularinterpretationofP(butnottheonlyinterpretation)isasalongrunfrequency:inaverylargenumber(N)ofrepetitionsoftheexperiment,(numberoftimesEoccurs)/N≈PE,providedtheexperimentsareindependentofeachother.Asmanyauthorshavepointedout,thereissomethingfishyaboutthisinterpretation.Forexam-ple,itisdifficulttomakeprecisethemeaningof“independentofeachother”withoutresortingtoexplanationsthatdegenerateintocirculardiscussionsaboutthemeaningofprobabilityandin-dependence.Thisfactdoesnotseemtotroublemostsupportersofthefrequencytheory.Theinterpretationisregardedasajustificationfortheadoptionofasetofmathematicalrules,orax-ioms.Thefirstfourrulesareeasytorememberifyouthinkofprobabilityasaproportion.Onemorerulewillbeaddedsoon.Statistics241:1September1997c°DavidPollardChapter1 Probabilitiesandrandomvariables Page2Rulesforprobabilities(P1):0≤PE≤1foreveryeventE.(P2):Fortheemptysubset∅(=the“impossibleevent”),P∅=0,(P3):Forthewholesamplespace(=the“certainevent”),PS=1.(P4):IfaneventEisadisjointunionofeventsE1,E2,...thenPE=PiPEi.<1.1> Example. FindP{atleasttwoheads}forthetossingofthreecoins.Usethesamplespacefromthepreviouspage.Ifweassumethateachcoinisfairandthattheoutcomesfromthecoinsdon’taffecteachother(“independence”),thenwemustconcludebysymmetry(“equallylikely”)thatP{hhh}=P{hht}=...=P{ttt}.ByruleP4theseeightprobabilitiesaddtoPS=1;theymusteachequal1/8.AgainbyP4,P{atleasttwoheads}=P{hhh}+P{hht}+P{hth}+P{thh}=1/2.¤Probabilitytheorywouldbeveryboringifallproblemsweresolvedlikethat:breaktheeventintopieceswhoseprobabilitiesyouknow,thenadd.Thingbecomemuchmoreinterestingwhenwerecognizethattheassignmentofprobabilitiesdependsuponwhatweknoworhavelearnt(orassume)abouttherandomsituation.Forexample,inthelastproblemwecouldhavewrittenP{atleasttwoheads|coinsfair,“independence,”...}=...toindicatethattheassignmentisconditionaloncertaininformation(orassumptions).Theverti-calbarisreadasgiven;werefertotheprobabilityof...giventhat...Forfixedconditioninginformation,theconditionalprobabilitiesP{...|info}satisfy•conditionalprobabilitiesrules(P1)through(P4).Forexample,P¡∅|info¢=0,andsoon.Iftheconditioninginfor-mationstaysfixedthroughouttheanalysis,oneusuallydoesn’tbotherwiththe“given...”,butiftheinformationchangesduringtheanalysisthisconditionalprobabilitynotationbecomesmostuseful.Thefinalrulefor(conditional)probabilitiesletsusbreakoccurrenceofaneventintoasuccessionofsimplerstages,whoseconditionalprobabilitiesmightbeeasiertocalculateorassign.Oftenthesuccessivestagescorrespondtotheoccurrenceofeachofasequenceofevents,inwhichcasethenotationisabbreviated:P¡...|eventAhasoccurredandpreviousinfo¢orP¡...|A∩previousinfo¢where∩meansintersectionorP¡...|A,previousinfo¢orP¡...|A¢ifthe“previousinfo”isunderstood.Thecommainthethirdexpressionisopentomisinterpretation,butitsconveniencerecommendsit.Imustconfesstosomeinconsistencyinmyuseofparenthesesandbraces.Ifthe“...”isade-scriptioninwords,then{...}denotesthesubsetofSonwhichthedescriptionistrue,andP{...}orP{...|info}seemsthenaturalwaytodenotetheprobabilityattachedtothatsubset.How-ever,ifthe“...”standforanexpressionlikeA∩B,thenotationP(A∩B)orP¡A∩B|info¢looksnicertome.Itishardtomaintainaconventionthatcoversallcases.Youshouldnotat-tributemuchsignificancetodifferencesinmynotationinvolvingachoicebetweenparenthesesandbraces.Statistics241:1September1997c°DavidPollardChapter 1 Probabilities and random variables Page 3Rule for conditional probability(P5) : if A and B are events thenP¡AB | info¢= P¡A | info¢· P¡B | A, info¢.The frequency interpretation might make it easier for you to appreciate this rule. Suppose that inN “independent” repetitions (given the same initial conditioning information)A occurs NAtimes,A ∩ B occurs NA∩Btimes.Then, for big N,P¡A | info¢≈ NA/NP¡A ∩ B | info¢≈ NA∩B/N.If we ignore those repetitions where A fails to occur then we have NArepetitions given the orig-inal information and occurrence of A,inNA∩Bof which B occurs. Thus P¡B | A, info¢≈NA∩B/NA. The rest is division.<1.2> Example. What is the probability that a hand of 5 cards contains four of a kind?Let us assume everything fair and aboveboard, so that simple probability calculations can be car-ried out by appeals to symmetry. The fairness assumption could be carried along as part of theconditioning information, but it would just clog up the notation to no useful purpose.Start by breaking the event of interest into 13 disjoint pieces:{four of a kind}=13[i=1FiwhereF1={four aces, plus something else},F2={four twos, plus something else},...F13={four kings, plus something else}.By symmetry each Fihas the same probability, which means we can concentrate on just one ofthem. By rule P4,P{four of a