CS 501: Software EngineeringAdministrationLectures on Reliability and DependabilityReliabilityUser Perception of ReliabilityReliability MetricsReliability Metrics for Distributed SystemsRequirements Specification of System ReliabilityCost of Improved ReliabilityExample: Central Computing SystemSlide 11Slide 12Building Dependable Systems: Three PrinciplesDependable and Reliable Systems: The Royal MajestyThe Royal Majesty: AnalysisThe Royal Majesty: Software LessonsReliability: Modified Waterfall ModelKey Factors for Reliable SoftwareBuilding Dependable Systems: Organizational CultureBuilding Dependable Systems: ComplexityBuilding Dependable Systems: Specifications for the ClientBuilding Dependable Systems: Quality Management ProcessesBuilding Dependable Systems: ChangeReviews: Process (Plan)Reviews: Design and CodeBenefits of Design and Code ReviewsReview Team (Full Version)Example: Program DesignReview ProcessStatic and Dynamic VerificationStatic Validation & VerificationStatic Verification: Program InspectionsInspection Checklist: Common ErrorsStatic Analysis ToolsStatic Analysis Tools (continued)1CS 501 Spring 2006CS 501: Software EngineeringLecture 19Reliability 12CS 501 Spring 2006Administration3CS 501 Spring 2006Lectures on Reliability and DependabilityLecture 19, Reliability 1: The development processReviewsLecture 20, Reliability 2: Different aspects of reliabilityProgramming techniquesLecture 21, Reliability 3: Testing and bug fixingTools4CS 501 Spring 2006ReliabilityReliability: Probability of a failure occurring in operational use.Perceived reliability: Depends upon: user behavior set of inputs pain of failure5CS 501 Spring 2006User Perception of Reliability1. A personal computer that crashes frequently v. a machine that is out of service for two days.2. A database system that crashes frequently but comes back quickly with no loss of data v. a system that fails once in three years but data has to be restored from backup.3. A system that does not fail but has unpredictable periods when it runs very slowly.6CS 501 Spring 2006Reliability MetricsTraditional Measures• Mean time between failures• Availability (up time)• Mean time to repairMarket Measures• Complaints• Customer retentionUser Perception is Influenced by• Distribution of failuresHypothetical example: Cars are less safe than airplanes in accidents per hour, but safer in accidents per mile.7CS 501 Spring 2006Reliability Metrics for Distributed SystemsTraditional metrics are hard to apply in multi-component systems:• In a big network, at any given moment something will be giving trouble, but very few users will see it.• A system that has excellent average reliability might give terrible service to certain users.• There are so many components that system administrators rely on automatic reporting systems to identify problem areas.8CS 501 Spring 2006Requirements Specification of System ReliabilityExample: ATM card readerFailure class Example MetricPermanent System fails to operate 1 per 1,000 daysnon-corrupting with any card -- rebootTransient System can not read 1 in 1,000 transactionsnon-corrupting an undamaged cardCorrupting A pattern of Never transactions corrupts database9CS 501 Spring 2006Cost of Improved Reliability$Up time99%100%Will you spend your money on new functionality or improved reliability?10CS 501 Spring 2006Example: Central Computing System A central computer serves the entire organization. Any failure is serious.Step 1: Gather data on every failure • 10 years of data in a simple data base• Every failure analyzed:hardwaresoftware (default)environment (e.g., power, air conditioning)human (e.g., operator error)11CS 501 Spring 2006Example: Central Computing System Step 2: Analyze the data• Weekly, monthly, and annual statisticsNumber of failures and interruptionsMean time to repair• Graphs of trends by component, e.g.,Failure rates of disk drivesHardware failures after power failuresCrashes caused by software bugs in each module12CS 501 Spring 2006Example: Central Computing System Step 3: Invest resources where benefit will be maximum, e.g.,• Orderly shut down after power failure• Priority order for software improvements• Changed procedures for operators• Replacement hardware13CS 501 Spring 2006Building Dependable Systems: Three PrinciplesFor a software system to be dependable:• Each stage of development must be done well.• Changes should be incorporated into the structure as carefully as the original system development.• Testing and correction do not ensure quality, but dependable systems are not possible without systematic testing.14CS 501 Spring 2006Dependable and Reliable Systems: The Royal MajestyFrom the report of the National Transportation Safety Board:"On June 10, 1995, the Panamanian passenger ship Royal Majesty grounded on Rose and Crown Shoal about 10 miles east of Nantucket Island, Massachusetts, and about 17 miles from where the watch officers thought the vessel was. The vessel, with 1,509 persons on board, was en route from St. George’s, Bermuda, to Boston, Massachusetts.""The Raytheon GPS unit installed on the Royal Majesty had been designed as a standalone navigation device in the mid- to late1980s, ...The Royal Majesty’s GPS was configured by Majesty Cruise Line to automatically default to the Dead Reckoning mode when satellite data were not available."15CS 501 Spring 2006The Royal Majesty: Analysis• The ship was steered by an autopilot that relied on position information from the Global Positioning System (GPS).• If the GPS could not obtain a position from satellites, it provided an estimated position based on Dead Reckoning (distance and direction traveled from a known point).• The GPS failed one hour after leaving Bermuda.• The crew failed to see the warning message on the display (or to check the instruments).• 34 hours and 600 miles later, the Dead Reckoning error was 17 miles.16CS 501 Spring 2006The Royal Majesty: Software LessonsAll the software worked as specified (no bugs), but ...• Since the GPS software had been specified, the requirements had changed (stand alone system to part of integrated system).• The manufacturers of the autopilot and GPS adopted different design philosophies about the communication of mode changes.• The autopilot was not programmed to recognize valid/invalid status bits in
View Full Document
Unlocking...