Chapter 12Biases in Estimating ProbabilitiesLujin WangPART THREE--COGNITIVE BIASES Biases in Estimating Probabilities• Simplified Rules: Availability Rule, Anchoring– Depended upon in making rough probability judgments, greatly ease the burden of decision• Expression of Uncertainty– A common source of ambiguity• Probability of a Scenario– Often miscalculated• Base-Rate Fallacy– Data on "prior probabilities“ are commonly ignored unless they illuminate causal relationshipsAvailability Rule• Used to make judgments about likelihood or frequency– “Availability" refers to imaginability or retrievability from memory• Two cues people use unconsciously in judging the probability of an event– The ease with which they can imagine relevant instances of the event – The number or frequency of such events that they can easily remember• Example:¾ We estimate our chances for promotion by recalling instances of promotion among our colleagues in similar positions and with similar experience.Availability Bias• Factors that influence judgment can be unrelated to the true probability of an event– How recently the event occurred– Whether we were personally involved– Whether there were vivid and memorable details in the event– How important it seemed at the time– The act of analysis itself• Example:¾ There are two CIA officers, one of whom knew Aldrich Ames and the other who did not personally know anyone who had ever turned out to be a traitor.Question: Which one is likely to perceive the greatest risk of insider betrayal?Availability Bias Influence• Likely to be Influenced by the availability bias – Analysts make quick judgments without really analyzing the situation (e.g. policymakers and journalists)• Less influenced by the availability bias– Intelligence analysts are evaluating all available information, not making quick and easy inferences• Intelligence analysts need to – Be aware when they are taking shortcuts– Know the strengths and weaknesses of these procedures– Be able to identify when they are most likely to be led astrayAnchoring• A natural starting point serves as an anchor– From a previous analysis of the same subject or from some partial calculation– Used as a first approximation to the desired judgment– Adjusted, based on the results of additional information or analysis• Problem– The starting point reduces the amount of adjustment– The final estimate remains closer to the starting point than it ought to beAnchoring Bias• Example:¾ Asking a group of students to estimate the percentage of member countries in the United Nations that are located in Africa.¾ Give half the students a low-percentage number and half a high-percentage number as an estimated answer. Then they adjust this number until they get as close as possible to what they believe is the correct answer.¾ Experiment results:¾ Problem: Starting points acted as anchors, causing drag or inertia that inhibited full adjustment of estimates45 percent65 percent25 percent10 percentAveraged EstimateAnchor (Starting point)Avoid Anchoring Bias• Reasons for the anchoring phenomenon are not well understood• If the estimated range is based on relatively hard information concerning the upper and lower limits, the estimate is likely to be accurate• Techniques for avoiding the anchoring bias– Ignore one's own or others' earlier judgments and rethink a problem from scratch– Employ formal statistical proceduresExpression of Uncertainty• Probabilities may be expressed in two ways– Statistical probabilities are based on empirical evidence– Make a "subjective probability" or "personal probability" judgment• Verbal expressions of uncertainty– "possible," "probable," "unlikely," "may," and "could“– Sources of ambiguity and misunderstanding– Make it easier for a reader to interpret a report as consistent with the reader's own preconceptions• Analysts must learn to routinely communicate uncertainty using the language of numerical probability or odds ratiosAssessing Probability of a Scenario• Intelligence analysts sometimes present judgments in the form of a scenario– a series of events leading to an anticipated outcome• The probability of a scenario should be the multiplication of the probabilities of each individual event• Approaches used to simplify the estimate lead problem– To assume one or more probable events have occurred– Averaging strategy: to base judgment on a rough average of the probabilities of each event • This violates the principle that a chain cannot be stronger than its weakest link• The least probable event in a scenario sets the upper limit on the probability of the scenario as a wholeBase-Rate Fallacy• In assessing a situation, an analyst sometimes has two kinds of evidence available– Specific evidence about the individual case at hand– Numerical data that summarize information about many similar cases• This type of numerical information is called a base rate or prior probability• The base-rate fallacy is that the numerical data are commonly ignored unless they illuminate a causal relationshipBase-Rate Fallacy• Example:¾ A fighter plane made a strafing attack on a US aerial mission attwilight. Both Cambodian and Vietnamese jets operate in the area. You know the following facts:(a) Specific case information: The US pilot identified the fighter as Cambodian. The pilot's aircraft recognition capabilities: thepilot made correct identifications 80% of the time and erred 20%of the time.(b) Base rate data: 85% of the jet fighters in that area are Vietnamese; 15% are Cambodian.¾ Question: What is the probability that the fighter was Cambodianrather than Vietnamese?¾ Answer: (15% x 80%) / (15% x 80% + 85% x 20%) = 12/29 =