Decision Analysis & Decision Support 6.872/HST.950Tasks? Mechanics Record keeping Administration Scheduling … Diagnosis Prognosis TherapyTypes of Decision Support • “Doctor's Assistant” for clinicians at any level of training • Expert (specialist) consultation for non-specialists • Monitoring and error detection • Critiquing, what-if • Guiding patient-controlled care • Education and Training • Contribution to medical research • …Two Historical Views on How to Build Expert Systems Great cleverness Powerful inference abilities Ab initio reasoning Great stores of knowledge Possibly limited ability to infer, but Vast storehouse of relevant knowledge, indexed in an easy-to-apply formChange over 30 years • 1970’s: human knowledge, not much data • 2000’s: vast amounts of data, traditional human knowledge (somewhat) in doubt • Could we “re-discover” all of medicine from data? I think not! • Should we focus on methods for reasoning withuncertain data? Absolutely! • But: Feinstein, A. R. (1977). “Clinical BiostatisticsXXXIX. The Haze of Bayes, the Aerial Palaces ofDecision Analysis, and the Computerized OuijaBoard.” Clinical Pharmacology and Therapeutics 21: 482-496.Cancer Test • We discover a cheap, 95% accurate test for cancer. • Give it to “Mrs. Jones”, the next person who walks by 77 Mass Ave. • Result is positive. • What is the probability that Mrs. Jones has cancer?Figuring out Cancer Probability Assume Ca in 1% of general population: + 950 1,000 Ca 95% -50 100,000 + 4,950 99,000 95% -94,050At the Extremes • If Ca probability in population is 0.1%, – Then post positive result, p(Ca)=1.87% • If Ca probability in population is 50%, – Then post-positive result, p(Ca)=95%Bayes’ Rule + + --Odds/Likelihood FormDeDombal, et al. Experience 1970’s & 80’s • “Idiot Bayes” for appendicitis • 1. Based on expert estimates -- lousy • 2. Statistics -- better than docs • 3. Different hospital -- lousy again • 4. Retrained on local statistics -- goodRationality • Behavior is a continued sequence of choices, interspersed by the world’s responses • Best action is to make the choice with the greatest expected value • … decision analysisExample: Gangrene • From Pauker’s “Decision Analysis Service” at New England Medical Center Hospital, late 1970’s. • Man with gangrene of foot • Choose to amputate foot or treat medically • If medical treatment fails, patient may die or may have to amputate whole leg. • What to do? How to reason about it?Decision Tree for GangreneEvaluating the Decision Tree 880 841.5 871.5 686 871.5 686 597Decision Analysis: Evaluating Decision Trees • Outcome: directly estimate value • Decision: value is that of the choice with the greatest expected value • Chance: expected value is sum of (probabilities x values of results) • “Fold back” from outcomes to current decision. • Sensitivity analyses often moreimportant than result(!)HELP System uses D.A. Warner HR, Computer-Assisted Medical Decision Making, Acad. Press 1979 Utility of operating minusUtility of not operating (∆u)4000300020002000+1000-10000Probability that patient has appendicitisEffect of age patient and MC (mortality for appendicitis without operation) onthe probability threshold (point of crossing zero ∆ u line) for decision to operate.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0MC = .014MC = .010MC = .014MC = .014MC = .008MC = .008Age 27Intellectual ToolsAge 57Image by MIT OpenCourseWare. Adapted from Warner, Homer R. "Computer-assisted medical decision making." Academic Press, 1979.Utility Analysis of Appendectomy Probability of appendicitisEffect of patient's salary and assumed value of one day of good health($70 or $140) on decision to operate for appendicitis.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.05000400030002000+1000-1000200030000Utility of operating minusUtility of not operating (∆µ)35000Value of 1 dayof good healthSalary140140140707070160008000350001600080004: Computer Representation of Medical KnowledgeImage by MIT OpenCourseWare. Adapted from Warner, Homer R. "Computer-assisted medical decision making." Academic Press, 1979.PROB OF APPENDICITIS A APPENDICITIS BY HISTORY B REBOUND TENDERNESS IN RLQ C PRIOR APPENDECTOMY D IF C THEN EXIT E WHITE BLOOD COUNT (WBCx100) TH/M3, LAST F PROB B A 620 90 G PROB F 43 18 9, 74 23 7, 93 18 11, 108 10 11, 121 16 13, 134 6 16, 151 5 16, 176 4 14 FVAL GUTILITY OF APPENDECTOMY IS ESTIMATED AS $--- - A (A) AGE B SEX C (A) SALARY, GET A/365 D JOB, PERCENT ACTIVITY NEEDED E LE A,B F DLOS D 30 1, 65 2, 80 4, 90 1, 100 – 0 G DLOS D 40 1, 80 4, 95 5, 100 – 0 … I COND E, F, 7, 1800, 0, C J COND E, G, 1, 900, 0, C … M PROB OF APPENDICITIS N UTIL M, I, J, K, L O IF N LT 0, EXIT FVAL N“Paint the Blackboards!” DECISION PATIENT STATE UTILITY Treat disease Disease (p) treat No disease (1-p) Treat no disease Disease (p) No treat disease No treat No disease (1-p) No treat no diseaseThreshold • Benefit B = U(treat dis) – U (no treat dis) • Cost C = U(no treat no dis) – U(treat no dis) • Threshold probability for treatment: Pauker, Kassirer, NEJM 1975Test/Treat Threshold Figure removed due to copyright restrictions. See Kassirer, Jerome P., and Stephen G. Pauker. "Should DiagnosticTesting be Regulated?" New England Journal of Medicine (1978).Visualizing Thresholds Figure removed due to copyright restrictions. See Kassirer, Jerome P., and Stephen G. Pauker. "Should DiagnosticTesting be Regulated?" New England Journal of Medicine (1978).More Complex Decision Analysis Issues • Repeated decisions • Accumulating disutilities • Dependence on history • Cohorts & state transition models • Explicit models of time • Uncertainty in the uncertainties • Determining utilities – Lotteries, … • Qualitative modelsExample: Acute Renal Failure • Based on Gorry, et al., AJM 55, 473-484, 1973. • Choice of a handful (8) of therapies (antibiotics, steroids, surgery, etc.) • Choice of a handful (3) of invasive tests (biopsies, IVP, etc.) • Choice of 27 diagnostic “questions” (patient characteristics, history, lab values, etc.) • Underlying cause is one of 14 diseases – We assume one and only one diseaseDecision Tree for ARF • Choose: – Surgery for obstruction – Treat with antibiotics – Perform pyelogram – Perform arteriography