Therac 25 Nancy Leveson University of Washington Clark S Turner UC Irvine Lecture 39 Keep the socio technical system in mind The Machine Hospitals and Clinics Video audio etc Doctors Medical Physicists Hardware Software Systems Management User groups Operators Reporting Patient Atomic Energy of Canada Limited Gov t Medical Device Regulation Management Reporting FDA Design teams sales staff support and field engineers Canadian Radiation Protection Bureau Reporting procedures www computingcases org The Machine The Therac 25 was created in the mid 1970s AECL reused code from the Therac 6 and Therac 20 Hardware safety checks replace by software Dual mode X ray and proton PDP 11 controls the turntable Tested in 1983 Assumed no design errors in software Considered software failures like computer selects wrong mode but assigned them probabilities of 4 x 10 9 There were a total of 6 cases reported 1 Katy Yarbrough constant pain lost the use of her arm and shoulder and had to have her breast removed Kennestone 2 Frances Hill cancer took her in November but she would have needed a full hip replacement Ontario 3 Woman develops erythema on her hip Yakima 4 Voyne Ray Cox died from overdose Tyler 5 Verdon Kidd died from radiation injury to his brain Tyler 6 Glen A Dodd died from complications of the radiation overdose Yakima Some of the problems The Machine Hospitals and Clinics No hardware checks for software safety Lack of training No computer in control of only turntable Technician not checking on patient in event of broken equipment Atomic Energy of Canada Limited Programming Errors Not integrating testing into development of software Poor HCI error messages Lack of documentation Lack of maintaining equipment Gov t Medical Device Regulation Inadequate requirements for reporting problems Not officially documenting Therac 25 Some solutions to the problems The Machine Hospitals and Clinics Install hardware checks like Therac 20 Pay for training or time reading manual Install a computer to control turntable Maintain equipment do not let patients use broken machines Test and Set Atomic Energy of Canada Limited More vigorous code reviews Integrate testing into development of software Usabilty test Interface Write more detailed documentation Gov t Medical Device Regulation Look at loopholes and adjust requirements for reporting problems Too late to officially document Therac 25 Abbott AIM Plus Incident 2006 a patient received a fatal overdose of fluorouracil a chemotherapy drug Nurse had to calculate the rate to be delivered 5 250 45 57 mL over 24X 4 hours Thimbleby H 2008 FEATUREIgnorance of interaction programming is killing people interactions 15 5 Sep 2008 52 57 DOI http doi acm org 10 1145 1390085 1390098 Abbott AIM Plus Incident 2006 a patient received a fatal overdose of fluorouracil a chemotherapy drug Nurse had to calculate the rate to be delivered 5 250 45 57 mL over 24X 4 hours 5 250 45 57 4 24 Nurse used a calculator and verified it with another nurse Thimbleby H 2008 FEATUREIgnorance of interaction programming is killing people interactions 15 5 Sep 2008 52 57 DOI http doi acm org 10 1145 1390085 1390098 Abbott AIM Plus Incident 2006 a patient received a fatal overdose of fluorouracil a chemotherapy drug Nurse had to calculate the rate to be delivered 5 250 45 57 mL over 24X 4 hours 5 250 45 57 4 24 Nurse used a calculator and verified it with another nurse Need a calculator with OR Need to know how to use memory functions Neither divided by 24 hours a day Gave patient 28 8 mL per hour instead of 1 2 mL More than 30 mL of fluorouracil per day should have raised warnings 1 mL a day is a high dose Alaris SE pumps pointing fingers FDA officials declined to name the maker of the infusion pumps In 2006 Cardinal Health of Dublin Ohio stopped production of its Alaris SE pumps because of a key bounce error that reportedly killed two patients including a 16 day old baby that got 44 8 milliliters of intravenous nutrition rather than 4 8 milliliters During the investigation into the malfunctioning pumps nurses complained about frequent keyboard errors while the manufacturer blamed nurses for entering the wrong drug information and then failing to double check said Brian Fitzgerald who heads the FDA s software specialists http paragraft wordpress com 2008 07 03 when bugs really do matter 22 years after the therac 25 Alaris SE pumps pointing fingers In an August 15 recall letter Alaris informed customers that it will provide a warning label for the pumps and a permanent correction for the key bounce problem once it is available http paragraft wordpress com 2008 07 03 when bugs really do matter 22 years after the therac 25 Alaris SE pumps removing the up key Proper StanceWhen programming pumps stand squarely in front of the keypad ideally with the pump at eye level for best visibility to facilitate proper depth of depressing each key ListenFocus on listening to the number of beeps while programming IV pumps each beep will correspond to a single digit entry Unexpected double tone could indicate an unintended entry Verify Screen Display Independent Double Check Look http paragraft wordpress com 2008 07 03 when bugs really do matter 22 years after the therac 25 Other Examples http www youtube com watch v brNbDWnHDVs eurl http interactions a cm org content p 1151 feature player embedded The Tyler Software Bug Focusing on particular software bugs is not the way to make a safe system Hand Keyboard Handler 0 Reset Data Entry Complete 1 Datent Treat 2 Set Up Done 1 3 Set Up Test Tphase Control Variable Offset Params Mode Energy 4 Patient Treatment Mode Energy Offset MEOS 5 Pause Treatment 6 Terminate Treatment Calibration Tables 7 Date Time ID Changes Interface Keyboard handler updates mode energy for Hand turntable Hand Keyboard Handler 0 Reset Data Entry Complete 1 Datent Treat 2 Set Up Done 1 3 Set Up Test Tphase Control Variable Offset Params Mode Energy 4 Patient Treatment Mode Energy Offset MEOS 5 Pause Treatment 6 Terminate Treatment Calibration Tables 7 Date Time ID Changes Interface Keyboard handler updates operating params for Datent Hand Keyboard Handler 0 Reset Data Entry Complete 1 Datent Treat 2 Set Up Done 1 3 Set Up Test Tphase Control Variable Offset Params Mode Energy 4 Patient Treatment Mode Energy Offset MEOS 5 Pause Treatment 6 Terminate Treatment Calibration Tables 7 Date Time ID Changes Interface Race Condition If Data Entry set before updating MEOS Datent will use old MOES info