hperry - 4/30/011Real Time System Testing - MIT 16.070hperry - 4/30/012Real Time System Testing - Lectures 30-32Why are we learning this?• A Real Time System must run continuously until it has been poweredoff• Failure to run properly can mean great economic loss and/or loss ofhuman lifeA LARGE PART OF REAL TIME SOFTWARE DEVELOPMENTMUST FOCUS ON AVOIDING FAILUREhperry - 4/30/013Real Time System Testing• The next three lectures will focus on:– Lecture 30: (R 11.3)• How to minimize failure in real time systems• Methods used to test real time systems– Lecture 31: (R 13)• What is Software Integration?• Test Tools• An example approach for integration and test of the MIT16.070 final project– Lecture 32: (R 11.4)• Fault Tolerance• Exception Handling• Formal Test Documentationhperry - 4/30/014Real Time System Testing - MIT 16.070Lecture 30hperry - 4/30/015Real Time System Testing• What are some reasons that Real Time software can fail?hperry - 4/30/016How do we minimize failure in real time systems?• Testing, Testing, Testing...– Software Test (various levels)– System Test• Build test and recovery software and/or hardware into the design– Exception handling (limit checking, etc.) to recover from bad datainputs– Redundant hardware solutions (e.g. 5 processors on Space Shuttle)– Fault Tolerant software and hardwarehperry - 4/30/017Real Time System TestingSWRequirementsSWDesignCodeTestsAcceptanceTests /DeliveryUser50%50%• Testing is an important part of the software life cycle • Testing can take up to 50% of a project’s budget and schedulehperry - 4/30/018Real Time System Testing• The goal of software testing a program is to find and fix errors prior todelivery to the end user• Testing:– Uncovers errors– Fixes errors– Measures requirements conformance– Provides an indication of quality• Testing a real time system is often difficult because of the very natureof real time systemshperry - 4/30/019Why Test? - Some Real World Examples• An F-16 pilot was sitting on the runway doing the pre-flight and wondering if thecomputer would let him raise the landing gear while on the ground….it did!• When initially developing the sidewinder missile mounting, there were a few problems.The software would release the latch and fire the missile. Initially, the latch was closedtoo quickly and the missile could never leave the wing. Imagine the pilot’s dismaywhen there was suddenly extra thrust on one of the wings!!• The F-16 has a sophisticated s/w system that performs load balancing to optimize flightperformance. This includes dropping empty fuel tanks in such a way as to balance theplane. A minor prerequisite to dropping the tank was overlooked in the software - it isusually a good idea to be upright when releasing the tanks. Imagine flying upside downand having empty fuel tanks come flying off!From a talk by Nancy Leveson, Dept. of CS and Eng, Univ of Washington onSW Safety and Reliability (IEEE and ACM sponsored 4/20/94)hperry - 4/30/0110Testing Considerations• Testing Techniques - the different approaches to testing software• Types of Tests - gaining different perspective from a suite of tests• Levels of Testing - determining how much granularity is needed?hperry - 4/30/0111Let’s apply what testing means in an example system:A remote controlled helicopter.A user can enter a number of commands to control the helicopter modes:- Take off- 30-degree right turn- 30-degree left turn- Forward flight- Hover- LandThe helicopter monitors its fuel remaining and communicates its mission time remaining back to the laptop.Communication between the vehicle and the laptop is handled by a radio frequency (RF) modem.The helicopter must work autonomously ifthe laptop communication link is lost.To implement this, we need:User interface software on laptop Assume commercial OS on laptopCommunication sw for RF modem (driver) on bothComm mgr (decodes messages for application) on bothEngine control software to process commands on helicopterFuel moni toring software on helicopterTask scheduler on helicopterhperry - 4/30/0112Remote Controlled Helicopter ExampleRFModemRF ModemCommunicationsManagerHelicopterEngineControlFuel MonitorCommunicationsManagerUser InterfaceLaptopTask SchedulerA/DfuelActualmodeTime remainingEngineDesiredmodehperry - 4/30/0113Testing Techniques• Black Box Testing– Only inputs and outputs of functions are considered–How outputs are generated based on a set of inputs is ignored– Run a suite of test cases• Exhaustive combination of all inputs• Corner cases (min, max, avg)• Pathological cases (inputs likely to result in error)– Disadvantage: Often bypasses unreachable code• Helicopter Example: Fuel Monitor– Test to verify mission time remaining outputs given varied fuellevel inputs and helicopter modes.hperry - 4/30/0114Testing Techniques• White Box Testing– Exercises all paths in a module– Driven by logic– Static Example: Code Inspections• group walkthrough of software logic• inspect code line-by-line– Dynamic Example: Test all the links and buttons on a web page• Helicopter Example:– Test communication interface between laptop and helicopter toconfirm all modes can be exercised.– Code inspection for Engine Controller to confirm appropriateengine controls for each helicopter modehperry - 4/30/0115Levels of Real Time System Testing• Unit Testing• Software Integration Testing• Software Validation and Verification Testing• Software / Hardware Integration Testing• System Testinghperry - 4/30/0116Unit Testing• Focuses on smallest unit of software (function, module, procedure)• Important control paths are tested• Usually developed by the software engineer who wrote the unit• Usually white-box oriented•Helicopter Example : Communication Manager Testing– Verify the Comm Mgr can parse data from the user interface andsend it to the RF modem– Exercise all paths within the Comm Mgr Software Unit(s)– Unit test could also include standalone tests of individual functionscalled by the Communication Managerhperry - 4/30/0117Software Integration Testing• Testing that occurs when unit tested modules are integrated into theoverall program structure• Test focuses on the interfaces between software modules• May be performed by developer or by independent test team• Black box testing perspective• Drivers and stubs will be required for external interfaces•Helicopter