Play TestingSoftware TestingTesting MethodGenerating Test SuitesSlide 5Combine Both ApproachesRandom TestingSlide 8Slide 9Team StructurePlay Testing TeamExternal TestersSlide 13External Testers: Half-LifeHalf-Life: Fine tuningHalf-LifeSlide 17AdviceMore AdviceMarch 24, Spring 2004 CS4455 1Play TestingSoftware TestingPlay Testing Team StructuresMarch 24, Spring 2006 CS 4455 2Software TestingGoal: “Prove” that the software works–Matches the functional specification–Matches the performance specificationReal Proof: NP-Hard problem–Simulate all the possible runs–Exponential growth in testing time –Each branch point multiplies possible program state by 2March 24, Spring 2006 CS 4455 3Testing MethodRun the program using a suite of data –Data suite exercises as much of the program’s code as possible–Each data set is accompanied by expected results–Where the results are unexpected, there is a bug!March 24, Spring 2006 CS 4455 4Generating Test SuitesBlack Box Testing–Generate test data based purely on interpretation of the specifications–Don’t look at the code!–Code is a Black BoxMarch 24, Spring 2006 CS 4455 5Generating Test SuitesWhite Box Testing–Use the code to generate tests with the purpose of exercising all blocks of codeCentral concept: CoverageCover all of the parts of the programs code –Ensure that all lines of code run at least onceMarch 24, Spring 2006 CS 4455 6Combine Both ApproachesBlack Box + White Box test suites –Complementary roles–Hard to tell from the data what approach was used–Aims to generate that ever-elusive “proof”Test suites:–Great for testing Compilers–Perhaps not so great for interaction and gamesMarch 24, Spring 2006 CS 4455 7Random TestingSome applications are hard to test:Random test–Give up the idea of formal coverage–Hope for stochastic coverage!Sometimes this depends on actual human interaction–Need to get lots of people–Beta testing by customersMarch 24, Spring 2006 CS 4455 8Play TestingWhat is it?March 24, Spring 2006 CS 4455 9Play TestingPlay testing follows software testing–Overlap of phasesSoftware Testing + More–Tuning game play–Tuning that Flow experience!Discover Intellectual Space–Does the player Get it??–Less important in established genresMarch 24, Spring 2006 CS 4455 10Team StructureFundamental issue: Willful blindnessDon’t fall in love with your code!Having testing team members be different from your coders is vitalA Software Testing teamA Play Testing TeamMarch 24, Spring 2006 CS 4455 11Play Testing TeamInternal team–Manages play-testing people–Does own play tests–Recruits outsiders for a week or twoExternal team–A population sample –Varying skills–Unfamiliar with product–Not in love with your product!–Not yet bored with the productMarch 24, Spring 2006 CS 4455 12External TestersDo they like the game?–They may lie to you–Politeness effect•If they can’t say why they like it, you have a problemDo they get frustrated?–What common areas?–Often skill-based•Internal team judges player’s skill, alsoMarch 24, Spring 2006 CS 4455 13External TestersFAQ–Be prepared to deal with the frequent questions in the final gameMarch 24, Spring 2006 CS 4455 14External Testers: Half-LifePeople near Valve’s offices who sent in registration cardsDesigners sit quietly while player struggles–Designer takes notesTypical 2-hour session–100 action itemsFirst 20-30 sessions absolutely vital–Learn what was fun–200 sessions totalMarch 24, Spring 2006 CS 4455 15Half-Life: Fine tuningAdd instrumentation–Player position, time, health, weapons–Activities:•Game save, dying, being hurt, solving a puzzle, fighting a monster…Graph series of sessions together–Spot too long with no encounters–Spot long periods of too much health–Spot long periods of too little healthMarch 24, Spring 2006 CS 4455 16Half-LifeMost important playtesting outcome:Clearly identified good and bad ideasAllowed people to…March 24, Spring 2006 CS 4455 17Half-LifeMost important playtesting outcome:Clearly identified good and bad ideasAllowed people to Abandon Bad IdeasPlaytesting provides the evidence neededMarch 24, Spring 2006 CS 4455 18AdviceDon’t get defensive!–The tester opinion is importantTesters: Stick to your findings!–Respectfully point out problemsMix up the hardware–Be honest about the specs on the boxMarch 24, Spring 2006 CS 4455 19More AdviceDiscourage Legacies–Don’t let bad decisions live