1!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 2 Lecture 10:!Managing Risk"General ideas about Risk"Risk Management"Identifying Risks"Assessing Risks"Case Study:"Mars Polar Lander"University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 3 Risk Management"About Risk"Risk is “the possibility of suffering loss”"Risk itself is not bad, it is essential to progress"The challenge is to manage the amount of risk"Two Parts:"Risk Assessment"Risk Control"Useful concepts:"For each risk: Risk Exposure"RE = p(unsatisfactory outcome) X loss(unsatisfactory outcome)"For each mitigation action: Risk Reduction Leverage"RRL = (REbefore - REafter) / cost of intervention"2!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 4 Likelihood of Occurrence Very likely Possible Unlikely (5) Loss of Life Catastrophic Catastrophic Severe (4) Loss of Spacecraft Catastrophic Severe Severe (3) Loss of Mission Severe Severe High (2) Degraded Mission High Moderate Low Undesirable outcome (1) Inconvenience Moderate Low Low Risk Assessment"Quantitative:"Measure risk exposure using standard cost & probability measures"Note: probabilities are rarely independent"Qualitative:"Develop a risk exposure matrix"Eg for NASA:"University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 5 Source: Adapted from Boehm, 1989 Top SE risks (with countermeasures)"Personnel Shortfalls" use top talent" team building" training"Unrealistic schedules/budgets" multisource estimation" designing to cost" requirements scrubbing"Developing the wrong software functions" better requirements analysis" organizational/operational analysis"Developing the wrong User Interface" prototypes, scenarios, task analysis"Gold Plating" requirements scrubbing" cost benefit analysis" designing to cost"Continuing stream of requirements changes" high change threshold" information hiding" incremental development"Shortfalls in externally furnished components" early benchmarking" inspections, compatibility analysis"Shortfalls in externally performed tasks" pre-award audits" competitive designs"Real-time performance shortfalls" targeted analysis" simulations, benchmarks, models"Straining computer science capabilities" technical analysis" checking scientific literature"3!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 6 Case Study: Mars Polar Lander"Launched"3 Jan 1999"Mission"Land near South Pole"Dig for water ice with a robotic arm"Fate:"Arrived 3 Dec 1999"No signal received after initial phase of descent"Cause:"Several candidate causes"Most likely is premature engine shutdown due to noise on leg sensors""University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 7 What happened?"Investigation hampered by lack of data"spacecraft not designed to send telemetry during descent"This decision severely criticized by review boards"Possible causes:"Lander failed to separate from !cruise stage (plausible but unlikely)"Landing site too steep (plausible)"Heatshield failed (plausible)"Loss of control due to dynamic effects (plausible)"Loss of control due to center-of-mass shift (plausible)"Premature Shutdown of Descent Engines (most likely!)"Parachute drapes over lander (plausible)"Backshell hits lander (plausible but unlikely)"4!University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 8 Premature Shutdown Scenario"Cause of error"Magnetic sensor on each leg senses touchdown"Legs unfold at 1500m above surface"software accepts transient signals on touchdown sensors during unfolding"Factors"System requirement to ignore the transient signals"But the software requirements did not describe the effect"Engineers present at code inspection didnʼt understand the effect"Not caught in testing because:"Unit testing didnʼt include the transients"Sensors improperly wired during integration tests (no touchdown detected!)"Result of error"Engines shut down before spacecraft has landed"estimated at 40m above surface, travelling at 13 m/s"estimated impact velocity 22m/s (spacecraft would not survive this)"nominal touchdown velocity 2.4m/s "University of Toronto Department of Computer Science © 2012 Steve Easterbrook. This presentation is available free for non-commercial use with attribution under a creative commons license. 9 FLIGHT SOFTWARE REQUIREMENTS3.7.2.2.4.2 Processinga. The lander flight software shall cyclically check thestate of each of the three touchdown sensors (one per leg)at 100 Hz during EDL.b. The lander flight software shall be able to cyclicallycheck the touchdown event state with or withouttouchdown event generation enabled.c. Upon enabling touchdown event generation, the lander flight software shall attempt to detect failed sensors bymarking the sensor as bad when the sensor indicates“touchdown state” on two consecutive reads.d. The lander flight software shall generate the landing event based on two consecutive reads indicatingtouchdown from any one of the“good” touchdownsensors..SYSTEM REQUIREMENTS1) The touchdown sensors shall be sampled at 100-Hz rate.The sampling process shall be initiated prior to lander entryto keep processor demand constant.However, the use of the touchdown sensor data shall notbegin until 12 meters above the surface.2) Each of the 3 touchdown sensors shall be testedautomatically and independently prior to use of thetouchdown sensor data in the onboard logic.The test shall consist of two (2) sequential sensor
View Full Document
Unlocking...