CS 501: Software EngineeringAdministrationSecurity and PeopleDesign for Security: PeopleSuggested ReadingValidation and VerificationThe Testing ProcessThe HeisenbugTest DesignTesting StrategiesMethods of TestingStages of TestingTesting: Unit TestingTesting: System and Sub-System TestingTesting:Acceptance TestingVariants of Acceptance TestingTest CasesTest Case Selection: Coverage of InputsTest Case Selection: ProgramTest Strategies: ProgramStatistical TestingSlide 22Regression TestingRegression Testing: Program TestingDocumentation of TestingA Note on User Interface TestingPowerPoint PresentationSlide 28Slide 29Slide 30Slide 31Slide 32Slide 33Fixing BugsMoving the Bugs Around1CS 501 Spring 2006CS 501: Software EngineeringLecture 21Reliability 32CS 501 Spring 2006Administration3CS 501 Spring 2006Security and PeoplePeople are intrinsically insecure:• Careless (e.g, leave computers logged on, use simple passwords, leave passwords where others can read them)• Dishonest (e.g., stealing from financial systems)• Malicious (e.g., denial of service attack)Many security problems come from inside the organization:• In a large organization, there will be some disgruntled and dishonest employees• Security relies on trusted individuals. What if they are dishonest?4CS 501 Spring 2006Design for Security: People• Make it easy for responsible people to use the system• Make it hard for dishonest or careless people (e.g., password management)• Train people in responsible behavior• Test the security of the system • Do not hide violations5CS 501 Spring 2006Suggested ReadingTrust in Cyberspace, Committee on Information Systems Trustworthiness, National Research Council (1999)http://www.nap.edu/readingroom/books/trust/Fred Schneider, Cornell Computer Science, was the chair of this study.6CS 501 Spring 2006Validation and VerificationValidation: Are we building the right product?Verification: Are we building the product right?In practice, it is sometimes difficult to distinguish between the two.That's not a bug. That's a feature!7CS 501 Spring 2006The Testing ProcessUnit, System and Acceptance Testing are major parts of a software project• It requires time on the schedule• It may require substantial investment in test data, equipment, and test software.• Good testing requires good people!• Documentation, including management and client reports, are important parts of testing.What is the definition of "done"?8CS 501 Spring 2006The Heisenbug9CS 501 Spring 2006Test DesignTesting can never prove that a system is correct. It can only show that (a) a system is correct in a special case, or (b) that it has a fault. • The objective of testing is to find faults.• Testing is never comprehensive.• Testing is expensive.10CS 501 Spring 2006Testing Strategies• Bottom-up testing. Each unit is tested with its own test environment.• Top-down testing. Large components are tested with dummy stubs.user interfaceswork-flowclient and management demonstrations• Stress testing. Tests the system at and beyond its limits.real-time systemstransaction processing11CS 501 Spring 2006Methods of TestingClosed box testingTesting is carried out by people who do not know the internals of what they are testing.Open box testing Testing is carried out by people who know the internals of what they are testing.(a) What is the advantage of each approach?(b) In each case, how do you set about selecting test cases?12CS 501 Spring 2006Stages of TestingTesting is most effective if divided into stagesUnit testing unit testSystem testing integration test function test performance test installation testAcceptance testing13CS 501 Spring 2006Testing: Unit Testing• Tests on small sections of a system, e.g., a single class• Emphasis is on accuracy of actual code• Test data is chosen by developer(s) based on their understanding of specification and knowledge of the unit• Can be at various levels of granularity • Open box: by the developer(s) of the unit If unit testing is not thorough, system testing becomes almost impossible. If your are working on a project that is behind schedule, do not rush the unit testing.14CS 501 Spring 2006Testing: System and Sub-System Testing • Tests on components or complete system, combining units that have already been thoroughly tested• Emphasis is on integration and interfaces• Uses trial data that is typical of the actual data, and/or stresses the boundaries of the system, e.g., failures, restart• Is carried out systematically, adding components until the entire system is assembled • Open or closed box: by development team or by special testersSystem testing is finished fastest if each component is completely debugged before assembling the next15CS 501 Spring 2006Testing:Acceptance Testing• Closed box: by the client• The entire system is tested as a whole• The emphasis is on whether the system meets the requirements• Uses real data in realistic situations, with actual users, administrators, and operatorsThe acceptance test must be successfully completed before the new system can go live or replace a legacy systemCompletion of the acceptance test may be a contractual requirement before the system is paid for16CS 501 Spring 2006Variants of Acceptance TestingAlpha Testing: Clients operate the system in a realistic but non-production environmentBeta Testing: Clients operate the system in a carefully monitored production environmentParallel Testing: Clients operate new system alongside old production system with same data and compare results17CS 501 Spring 2006Test CasesTest cases are specific tests that are chosen because they are likely to find faults.Test cases are chosen to balance expense against chance of finding serious faults.• Cases chosen by the development team are effective in testing known vulnerable areas.• Cases chosen by experienced outsiders and clients will be effective in finding gaps left by the developers.• Cases chosen by inexperienced users will find other faults.18CS 501 Spring 2006Test Case Selection: Coverage of InputsObjective is to test all classes of input• Classes of data -- major categories of transaction and data inputs.Cornell example: (undergraduate, graduate, transfer, ...) by (college, school, program, ...) by (standing) by (...)• Ranges of data -- typical values, extremes• Invalid
View Full Document
Unlocking...