ECE270: Handout 2Basic Concepts of Probability Theory (Part II)Outline:1. the three axioms of probability,2. basic events,3. basic events for finite S,4. basic events for countably infinite S,5. basic events for uncountably infinite S.F Three Axioms of Probability• The axioms are the foundations of the modern probability theory.• Probabilities are numbers assigned to events to indicate how likely it is that the events happenwhen we run the random experiment.• For every event E there exists a number P (E), referred to as the probability of E, whichsatisfies the following three axioms:1. 0 ≤ P (E) ≤ 1 i.e., given an event E, the probability that the experiment outcomeξ ∈ E is between 0 and 1.2. P (S) = 1 i.e., probability of event S is 1 (S is called certain event).3. Given the events E1, E2, ... are all mutually exclusive (i.e., Ei∩ Ej= Φ for all i, j, i 6= j)we have P (S∞i=1Ei) =P∞i=1P (Ei)if S is finite axiom 3 is equivalent to P (Sni=1Ei) =Pni=1P (Ei) (the number of eventsare finite for a finite S).• Some useful properties that can be proved using the three axioms are:1. P (Φ) = 0 i.e., probability of event Φ is 0 (Φ is called null or impossible event).2. P (Ec) = 1 − P (E)3. if E ⊂ F then P (E) ≤ P (F)4. P (E ∪ F ) = P (E) + P (F ) − P (E ∩ F )Exercise: Prove properties 1-4 using 1) set theory, 2) Venn diagrams.EXAMPLE 1Suppose a random experiment has the sample space S = {a, b, c, d}. Which of the followingsare TRUE?1) a ∈ S 2){a} ∈ S 3){a} ⊂ S 4){a, b} ⊂ {a, b, d} 5)P ({a, b}) ≤ P ({a, b, d})6) P ({b}) = P ({d}) 7)P (S) = 1 8){a, d} ∪ {b, a} = {a, b, d} 9)a ∪ b = a, b10) for two events A and B if A = B then P (A) = P (B)11) for two events A and B if P (A) = P (B) then A = BEXAMPLE 2 (axioms of probability)Suppose that A, B and C are three mutually exclusive events for which P (A) = 0.3, P (B) =0.5 and P (C) = 0.2. What is the probability that:– either A or B occurs.– A occurs but B does not.– either A or B or C occurs.– neither A nor B occurs.• So far, we have interpreted the probability of an event as being a measure of how frequentlyevent will occur, when the experiment is continually repeated. There are other uses of the termprobability (that are not a common perspective in engineering), e.g., personal or subjectiveview of probability, where probability is a measure of someone’s belief in the statementshe/she is making . Examples are the chance/probability that someone likes something, thechance/probability that a horse wins in a horse race, the chance/probability that it rainstomorrow. The three axioms and the above properties still apply.EXAMPLE 3 (personal or subjective view of probability)Jack is taking two books along on his holiday vacation. With probability 0.5 he will like thefirst book, with probability 0.4 he will like the second book. with probability 0.3 he will likeboth books. What is the probability that he likes neither book?• The three axioms and the above properties enable us to compute the probabilities of someevents in terms of probabilities of some other events.• What needs to be done is to identify basic events for each random experiment and assign prob-abilities to these basic events. Note that the probability assignments to the basic events mustsatisfy the three axioms. Given the probability of basic events, we can find the probability ofany other events.F Basic Events• The choice of basic events depend on type of S: 1) S is finite, 2) S is countably infinite, 3) Sis uncountably infinite.F Basic Events for Finite SPage 2• Basic events are elementary events.• Consider S = {ξ1, ξ2, ..., ξK} with elementary events Ek= {ξk} k = 1, ..., K. Any otherevent F, F ⊂ S can be decomposed as the union of some of the elementary events. Since theelementary events are mutually exclusive P (F ) is the sum of the probabilities of elementaryevents P (Ek) (by axiom 3). So we must assign probabilities to the elementary events Ekk =1, 2, ..., K such that:1) 0 ≤ P (Ek) ≤ 1 2)PKk=1P (Ek) = 1Exercise: Suppose the finite S has K elementary events. How many basic events does Shave? How many events does S have?• For finite S when the random experiment outcomes ξkare equally likely, then P (Ek) are allthe same and P (Ek) = 1/K for all k. In this case, the probability of an event is equal to thenumber of outcomes in the event divided by K.EXAMPLE 4 (finite S with not necessarily equally likely outcomes)Consider a random experiment with S = {0, 1, 2, 3} where P ({0}) = 1/2, P ({1}) = 1/4, P ({2}) =1/8. Let E = {2, 3} and F = {0, 3}. Find P (E) and P (F ).EXAMPLE 5 (finite S with equally likely outcomes)Suppose a fair coin is tossed three times and we observe the sequence of the heads and tails.Assuming the experiment outcomes are equally likely:a) what is the probability of having 2 heads in 3 tosses?b) what is the probability of having at least 2 heads in 3 tosses?EXAMPLE 6 (finite S with equally likely outcomes)Consider a random experiment with S = {0, 1, 2, 3} where P ({0}) = P ({1}) = P ({2}) =P ({3}). Let E = {2, 3} and F = {0, 3}. Find P (E) and P (F ).• Combinatorial analysis is the mathematical theory of counting and is very useful in findingprobabilities of events corresponding to random experiments with finite S when the numberof elements in S is very very large and (some of) the outcomes are equally probable (Chapter1 of Sheldon Ross text book).EXAMPLE 7 (combinatorial analysis)A committee of 5 is to be selected from a group of 6 men and 9 women. If the selection ismade randomly, what is the probability that the committee consists of 3 men and 2 women?Page 3Solution: We note that S is finite and the outcomes are equally probable. Using combina-torial analysis we have:P (committee consists of 3 men and 2 women) =0@631A0@921A0@1551A=20∗363003whereµnk¶=n!k!(n−k)!F Basic Events for Countably Infinite S• Basic events are elementary events. Consider S = {ξ1, ξ2, ...} with elementary events Ek={ξk} k = 1, 2, .... Any other event F, F ⊂ S can be written in terms of the elementary events.EXAMPLE 8Suppose we toss a fair coin until a heads appear. We observe how many times we flip thecoin.a) What is the probability that we flip a coin 3 times?b) What is the probability that we flip the coin at least 5 times?c) Can you generalize the results in parts a) and b)?F Basic Events for Uncountably Infinite S• Suppose S is, for example, real line, some intervals of the real line,