(Textbook PSLS 3ed. Pages 255-257) DISCUSSION Making sense of conditional probabilities in diagnostic tests Conditional probabilities are often grossly misinterpreted. Yet they are very important, particularly in medicine, where understanding test results requires understanding these probabilities. Many studies have documented that most physicians understand and communicate the true meaning of test results incorrectly. Here is a revealing quote from one such study: For instance, doctors with an average of 14 years of professional experience were asked to imagine using the Haemoccult test to screen for colorectal cancer. The prevalence [rate] of cancer was 0.3%, the sensitivity of the test was 50%, and the false positive rate was 3%. The doctors were asked: what is the probability that someone who tests positive actually has colorectal cancer? The correct answer is about 5%. However, the doctors’ answers ranged from 1% to 99%, with about half of them estimating the probability as 50% (the sensitivity) or 47% (sensitivity minus false positive rate). If patients knew about this degree of variability and statistical innumeracy they would be justly alarmed. The positive predictive value, PPV, describes the probability that a person who receives a positive test result actually has a given disease, P(disease| positive test). You can check that PPV in this study is indeed only (0.003)(0.5)(0.003)(0.5)+(0.097)(0.03)= 0.048, or about 5% The discrepancy between a perceived (incorrect) PPV and its actual value is particularly large for screening tests because they are typically administered to a large population with a relatively low rate of disease. When a disease is rare in the population screened, the percent of false-positives tends to be large, even when sensitivity and specificity are high. The consequence is a low PPV in the context of indiscriminate screening. (Note that these testing procedures have a much higher PPV when they are used to screen higher-risk individuals or to diagnose patients with a suspected condition based on existing symptoms.) The potential merit of screening is that early detection may play a key role in patient prognosis. However, the emotional, medical, and financial consequences of misinterpreted test results cannot be ignored. For instance, up until the summer of 2012, the U.S. Food and Drug Administration had refused approval to over-the-counter HIV screening because of concerns with public reaction to false-positives. Inversely, the 2009 recommendation by the U.S. Preventive Services Task Force against routine mammography screening for women in their 40s caused an uproar, despite the fact that the PPV in this age group is in the 10% to 20% range. Beyond debating the pros and cons of various diagnostic and screening tests, we should ask how such misunderstanding can happen. Physicians are highly trained professionals, and studies show that most do know the definitions of “sensitivity” and “specificity.” The challenge seems to be interpreting conditional probabilities, particularly confusing P(A| B) with P(B | A). In thestudy cited, many physicians confused the PPV, P (disease | positive test), with the test’s sensitivity, P (positive test | disease). Probability concepts tend to be more abstract than natural frequencies. Studies show that reframing information in a simpler, more natural format using a concrete example increases dramatically the proportion of physicians who correctly identify the PPV. The colorectal cancer case, for example, can be reframed as follows: Ten thousand patients take the Haemoccult test. Of these 10,000 patients, we expect that about 30 actually have colorectal cancer and the remaining 9970 do not. Of the 30 patients with colorectal cancer, about 15 (50%) can be expected to receive a positive test result. Of the 9970 patients without colorectal cancer, about 299 (3%) can also be expected to receive a positive test result. What proportion of patients who receive a positive test result can be expected to actually have colorectal cancer? The PPV seems much more intuitive now as the proportion of positive tests that are true-positives: PPV = 15/(15 + 299) = 0.048, or about 5%. Researchers indeed advocate providing information about diagnostic and screening tests in terms of natural frequencies in addition to giving the usual sensitivity and specificity conditional probabilities. Rather than being expected to do elaborate probability calculations on their own, physicians and patients should be given clear information about the predictive value of a test. This would improve medical practices and support informed patient decisions. Figure 10.9 for Exercise 10.44 on the PSA test illustrates a different and effective way to communicate probabilities for such tests. Until the general public understands these statistical concepts, we will continue to pay a high medical and financial price. Here is a most unusual and interesting illustration. In 2006, a Texas judge uncovered an extensive legal scam in which screening tests were used to fuel massive class action asbestos and silicone lawsuits. Instead of representing workers who sought them out because of existing health problems, some lawyers had advertised widely to the general public and indiscriminately tested very large numbers of people, including some with existing diagnoses for unrelated lung diseases. This resulted, obviously, in large numbers of false-positives. . . and lots of money for the lawyers and screening companies. The many workers who tested positive received a meager check in the mail and were never informed that a positive test could be a false-positive. Interestingly, the scheme was uncovered by a judge who had been a nurse earlier in her career. Judge Janis Jack was the first person, in years of such litigation, to ever ask how the plaintiffs had been screened. Comment: PPV depends on the prevalence (rate) of disease in the population screened because it influences how many true-positives and false-positives we would get. But the sensitivity and specificity are properties of the test kit that depends on only on chemistry and