CS 350: Introduction to Software EngineeringSurvey resultsWhy GCS (Guidance and Control Software)?Comments on Real TimeSoftware ExperimentationN-version SoftwareFailure Probabilities - 1Failure Probabilities - 2Failure Probability CommentsN-version Software - 2Oracle Problem - 1Oracle problem - 2GCS GoalGCS Context - 2GCS Context - 3Bigger Context, DesignReading the spec - 1Reading the spec - 2Reading the spec - 3Format of GCS Soft Req. DocData Element DictionaryQuizViking LanderViking OrbiterGCS SystemPhysical Systems (cont.)Slide 27GCS_SIM_RENDEZVOUS PurposeNASA Testing Architecture:Testing arch (cont.)Testing arch. (cont.)Specification IntentSome NotationsProcesses (typical real-time sys)Process (cont.)TimingA Little Bit of FORTRANLevel 0 Spec (context diagram)Level 1 SpecLevel 2 SpecLevel 3 Spec Formatcp process stepsAssignmentProgramming AssignmentEventual ProjectOther requirementsCS 350: Introduction toSoftware EngineeringSlide Set 3C. M. OverstreetOld Dominion UniversityFall 2004Survey resultsBest language:C++: Java:depends on task: lisp:Preferred language:C++: Java:Max size program you’ve written:< 500> 500 & < 1k:> 1k & < 10k:> 100kWhy GCS (Guidance and Control Software)?NASA & FAA interest in “ultra-reliable” softwareNASA/Langley lead in previous project (still core of knowledge about project)Complex but not too complexI like it because:Real-time systemApplication knowledge required (just like real life)Really messy in places (just like real life)~100 pg. req. doc.Bad features (just like real life)I’d do a bunch of things differently but we’re stuck with other people’s bad decisionsComments on Real TimeIn a real-time system:Computational tasks must complete by a deadlineOften cyclical: computations repeat at a specified frequencyOperating system considerationsMany OS’s allow you to schedule the start time for a task (see cron in UNIX). Much harder to guarantee that a task will complete by a deadline, particularly for interrupt driven systemsSoftware ExperimentationStd Problem is SE: many opinions, few factsPurpose of GCS: get some real dataStudy a software systems that:Doesn’t cost too much to buildSmall but not too smallIs in the “right” application domainFlight control, real timeCan be used for data collectionWant failure rate data (e.g. MTTF)Want to understand kinds of errors people makeN-version SoftwarePurpose: increase software reliabilityOne idea:Develop one requirements documentGive copies to several different development groups (say 5)Each group develops complete systemIndependently! No info exchange!!Run all five systems:Give each identical inputWait for each to produce required outputCompare 5 outputs & take majority as the correct outputQuestion: does this work?Failure Probabilities - 1In software reliability world, failures are treated as "random" events (even though they aren't)Each programs has "input space":The set of data the program will receive when running, including the fact that it will see some inputs frequently, others rarely.Failure Probabilities - 2Then probability of program failure is based on idea:Randomly pick a data value from the program's input space (where selecting particular values reflects how frequently they will be seen when the program really runs).Observe whether or not the program failsDo this for all values in input space (properly weighted).Pr(failure with a randomly selected input) = (number failures) / (number tests)Failure Probability CommentsSimple concept. But involves several potentially hard problems:What is the distribution of inputs the program will really encounter in use?Input space often too large to run all program with all possible inputsHow do you tell when the program fails (assuming it doesn't crash)?Typical industry interest is how often software will fail when used by their customers. This is a similar idea.Depends on both frequency of use and typical input dataN-version Software - 2Statistical concept: event independenceKnowing one event occurred doesn’t change probably of another event.Independence Pr(A^B)=Pr(A)Pr(B)Often, this is not trueKey Question:Do separately developed versions really fail independently?Or if one fails, does that make it more likely that another will fail also?Oracle Problem - 1When testing software, how do you detect all erroneous outputs?Unsolved SE problem for many applicationsOracle problem - 2Federal Aviation Administration (FAA) specifies that the failure rate for commercial aircraft of less than 10-9.How many test cases should you run to determine this?Generally considered a statistical problem: if the software is given a "random" input, what is the likelihood of the program producing incorrect output (assuming it doesn't crash)?How do you check the answers?GCS GoalGenerate data for software reliability studyWhat types of mistakes to do programmers make?How often do different programmers make the same mistake?If “approved” software development procedures are used (here, RTCA/DO-178B--required by FAA), how reliable will the software be?GCS Context - 2Use n-versions to address the oracle problem:Assume, say, 5 complete versions of GCS are available.Give each a set of inputs.After all finish, compare outputs.If identical repeat (run another test)If at least one differsWe’ve found an errorDon’t know which program.May be a problem with ambiguous requirements.Does this solve the oracle problem?GCS Context - 3Execution context:V1V2V3 V4Sensor DataVoter/LoggerPhysics, Environ.SimulatorEngine CommandsSensorDataSeparateGCS versionsV2Bigger Context, DesignViking project was largeDone by teams, different teams for different phasesMain phases:Launch rocket (with Viking lander as payload)Fly payload to Mars, place into orbitStart decent of payload to target location; parachute deployedControl descent to surface of MarsConduct scientific mission & transmit data to EarthAnalyze data collectedOur part!Reading the spec - 1Read pp. 3-18, 89-123 for Thurs.Read as reference doc. That is, most important is to be aware of what is there and where there rather than to memorize lots of details.Point: when you get to code design, you will (we hope) remember that
View Full Document