Testing Distributed SoftwareOutlineBibliographyDefects specific to concurrent & distributed swBasic concepts review – thread in BinderBasic concepts review -- threadComputational Models -- concurrentComputational Models -- networkedComputational Models -- (truly) distributedHow are distributed models different?NondeterminismNondeterminism -- continuedSlide 13Additional InfrastructurePartial failuresTime-OutsDynamic Nature of the StructureThreadsSpecifying synchronizationDefining useful pathsSignificant paths in distributed softwareWhat type of events are SYN-events?SYN-paths are necessary, not sufficientThread modelsDesign for TestabilityLife-cycle Testing in a distributed systemMore on life-cycle test planBasic Client/Server Model – simplest distributed systemCommercial infrastructuresCORBA - as a styleCORBA -- continuedCORBA style – testing implicationsRMI -- remote method invocationA Generic Distributed Component ModelReview of process communicationTest the RequesterProvider -- central roleThe many faces of the ProviderStubs and skeletonsStubs and SkeletonsSpecifying Distributed Objects -- IDLSpecifying Distributed Objects -- IDLSlide 43Pre- and Post conditions and InvariantsTest cases for implicit, ever-present errorsTemporal TestingTest EnvironmentSlide 48Model-Specific Test CasesModel-Specific Test Cases -- cont.More re Generic Distribution ModelSlide 52Slide 53Slide 54Slide 55Testing the InternetMcGregor & Sykes Don’t CoverTesting Distributed SoftwareECEN5053 Software Engineering of Distributed SystemsUniversity of Colorado, BoulderDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoOutlineTesting for Software Security – if timeTesting Patterns for Distributed Services - Binder’s bookDistributed Software Issues and approachesThreads and synchronizationPath testingLife-CycleModels and testing implicationsTest environmentDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoBibliographyRobert V. Binder, Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley Object Technology Series, c. 2000, ISBN 0-201-80938-9 (1,190+ pp.)John McGregor & David Sykes, A Practical Guide to Testing Object-Oriented Software, Addison-Wesley Object Technology Series, c. 2001, ISBN 0-201-32564-0 (393 pp.)Testing Applications On The WebClient-Server TestingHerbert H. Thompson & James Whittaker, Dr. Dobb’s Journal, “Testing for Software Security”, November 2002, pp. 24 - 32.December, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoDefects specific to concurrent & distributed swFailure to synchronize accesses to shared data values can lead to incorrect data values even though each thread is correctly computing its result.A specific node in a distributed system can fail to perform correctly even though every other node is performing properly.A network link between nodes can fail while the remainder of the system continues to function.December, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoBasic concepts review – thread in Binderthread -- independent context of execution within an operating system process. Has its own program counter and local dataSmallest unit of execution that can be scheduledMost operating systems allow a single process to group multiple threads into a related set that shares some properties and keeps others private.Sequence of computationsSimplest testing situation -- address various entry points and multiple paths throughMultiple threads complication -- share informationDependencies implies thread order mattersDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoBasic concepts review -- threadDeveloper must provide a synchronization mechanism to ensure correct orderOO languages provide some natural means of synchronization by hiding attributes behind interfacessometimes making threads correspond to objectssynchronization is visible in the object interfacemessaging is a key element in synchronizationClass testing would not likely detect synch. defectsObjects must interact to reveal a synch. defectDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoComputational Models -- concurrentDesign assumes multiple things are happening at the same timeTesting for concurrency defectsFocus on how two threads interactMethods should receive typical testing before being exercised in an interaction setting•State-driven testing, for exampleParallel processors in the same “box” -- we’re ignoring thisDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoComputational Models -- networkedPhysical concurrency achieved by linking together separate boxes with communication devicesCommunication devices operate at slower speed than the internal data busInternetDifficulty in synchronizing the many independent machinesTimes of events are measured by local clocksHard to determine thoroughness of testingDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoComputational Models -- (truly) distributedMultiple processes support a flexible architectureNumber of participating objects can changeObjects can be distributed across multiple processes on the same machineacross multiple physical computersComponents must be able to locate othersnaming service known to all componentsconfiguration file may list the machines authorized to participatepart of the infrastructure; may be standardized and reusable on many systems without modificationDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoHow are distributed models different?NondeterminismAdditional infrastructurePartial failuresTime-outsDynamic nature of the structureThese basic differences directly impact testing.December, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoNondeterminismDifficult to exactly replicate a test runDetermined by scheduler of the operating system(s)Changes in programs not associated with the SUT can affect the order in which threads of the SUT are executed.Just because a defect does not reoccur does not mean it has been repaired correctly.Leads to certain techniques -- next slideDecember, 2006Testing Distributed Software, ECEN5053, Univ. of ColoradoNondeterminism -- continuedMore thorough testing at the class levelReview should pursue whether adequate provision for synchronizationDynamic class testing should determine if synchronization is working correctly in a controlled test environmentExecute a large number of cases trying