1Spring 2008Spring 2008CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. 3D User Interface Evaluation II3D User Interface Evaluation IILecture #15: Evaluating 3DUIs Lecture #15: Evaluating 3DUIs ––Part IIPart IISpring 2008Spring 2008Joseph J. LaViola Jr.Joseph J. LaViola Jr.CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008Usability Evaluation in 3DUIsUsability Evaluation in 3DUIs2CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008Classification ShortcomingClassification ShortcomingDoes not tell you Does not tell you ““whenwhen””a method should a method should be appliedbe appliedDoes not tell you Does not tell you ““howhow””to apply more to apply more than one methodthan one method3DUI evaluation models3DUI evaluation modelsTestbedTestbedevaluationevaluationSequential evaluationSequential evaluationCAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation FrameworkEvaluation FrameworkDeveloped by Bowman and Hodges (1999)Developed by Bowman and Hodges (1999)Empirically evaluate techniques outside of Empirically evaluate techniques outside of applicationsapplicationsComponentsComponentsinitial evaluationinitial evaluationtaxonomytaxonomyoutside factorsoutside factorsperformance metricsperformance metricstestbedtestbedevaluationevaluationapplication and generalization of resultsapplication and generalization of results3CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluationEvaluationUser-centered Application8Heuristics&Guidelines7QuantitativePerformanceResults6T e s t b e dE v a l u a t i o n52TaxonomyOutside Factorstask, users, evnironment,system34PerformanceMetricsInitial Evaluation1CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––Initial EvaluationInitial EvaluationGain intuitive understanding of generic Gain intuitive understanding of generic interaction tasks and current technologiesinteraction tasks and current technologiesExperience and user observationExperience and user observationUsed for Used for building taxonomybuilding taxonomyidentifying outside factorsidentifying outside factorsfinding performance metricsfinding performance metrics4CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––TaxonomyTaxonomyDevelop Develop taxomonytaxomonyof interaction of interaction techniques for interaction task in questiontechniques for interaction task in questionCan use taskCan use task--subtask approachsubtask approachTaskTechniqueComponentSub-taskCAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––Outside FactorsOutside FactorsCannot evaluate in a vacuumCannot evaluate in a vacuumNeed to take other factors into accountNeed to take other factors into accountCategoriesCategoriestask characteristicstask characteristicsenvironment characteristicsenvironment characteristicsuser characteristics user characteristics system characteristicssystem characteristics5CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––MetricsMetricsObjective measuresObjective measuresspeed speed accuracyaccuracySubjective measuresSubjective measuresease of useease of useease of learningease of learningfrustrationfrustrationetcetc……CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––The The TestbedTestbedAllows generic, Allows generic, generalizablegeneralizable, and reusable , and reusable evaluationevaluationTestbedTestbedexamines all aspects of a taskexamines all aspects of a taskevaluates each technique componentevaluates each technique componentconsiders outside influencesconsiders outside influenceshas good metricshas good metricsNormally use formal, factorial experimental Normally use formal, factorial experimental designsdesigns6CAP6938 CAP6938 ––3D User Interfaces for Games and Virtual Reality 3D User Interfaces for Games and Virtual Reality ©©Joseph J. LaViola Jr. Joseph J. LaViola Jr. Spring 2008Spring 2008TestbedTestbedEvaluation Evaluation ––ResultsResultsProduces set of results or models that Produces set of results or models that characterize an interaction technique for a given characterize an interaction technique for a given tasktaskUsability in terms of multiple performance Usability in terms of multiple performance metricsmetricsResults become part of a performance database Results become part of a performance database for taskfor taskResults can be generalized into heuristics or Results