Usability Study DesignNew assignment!Conceptual Model of a SystemMismatched conceptual modelsFlying from Shanghai to JinanSlide 6Slide 7Hongqiao vs. Pudong airportsSlide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Current eventsCurrent events (2)TodayMethods for evaluating systemCritical incident techniqueQuestionnaires & surveysDesigning survey questionsLikert scalesSemi-structured interviewsQuantitative measures (comparison)Experiment design (2)Flow trackingUsage loggingCombining methodsCardinal ruleRecording the screenCamtasiaCamtasia demoEvaluating facebook applications?facebook demosFor your projectConsent formSlide 38Project statusPlanning for the demo!Presenting your work: demoDemoPosterSlide 44Slide 45Grading criteriaIndividual AssessmentsUsability study reportUsability Study Usability Study DesignDesignJohn C. TangJohn C. TangNovember 8, 2007November 8, 200722New assignment!New assignment!Travel to an unfamiliar countryTravel to an unfamiliar country–AloneAlone–Especially if you’re unfamiliar with Especially if you’re unfamiliar with the languagethe language–Record your first impressionsRecord your first impressions(only kidding about being an (only kidding about being an assignment!)assignment!)33Conceptual Model of a Conceptual Model of a SystemSystemDesign ModelDesign Model–The model the designer has The model the designer has of how the system worksof how the system worksSystem ImageSystem Image–How the system actually How the system actually works works –The structure and behavior The structure and behavior of the systemof the systemUser’s ModelUser’s Model–How the user understands How the user understands how the system workshow the system works44Mismatched Mismatched conceptual modelsconceptual models???55Flying from Shanghai Flying from Shanghai to Jinanto Jinan667788Hongqiao vs. Pudong Hongqiao vs. Pudong airportsairports991010111112121313141415151616Current eventsCurrent eventsOpenSocial APIsOpenSocial APIs–Profile Information Profile Information (user data)(user data)–Friends Information Friends Information (social graph)(social graph)–Activities Activities (things that happen, (things that happen, news feed type stuff)news feed type stuff)Focus on concepts, Focus on concepts, not implementationnot implementation1717Current events (2)Current events (2)Facebook adsFacebook ads–Business facebook Business facebook pagespages–Spreading marketing Spreading marketing messages through messages through social connectionssocial connections–Allowing businesses to Allowing businesses to gather facebook gather facebook activity data activity data –Further diluting the Further diluting the meaning of “friend”?meaning of “friend”?–Beware social phishingBeware social phishing1818TodayTodayReview usability study methodsReview usability study methodsApply to facebook applicationsApply to facebook applicationsReview actual usability study Review actual usability study reportreport1919Methods for evaluating Methods for evaluating system system QualitativeQualitative–Rich, subjectiveRich, subjective–Exploring conceptsExploring conceptsQuantitativeQuantitative–Precise, objective, repeatablePrecise, objective, repeatable–Demonstrating claimsDemonstrating claims2020Critical incident Critical incident techniquetechniqueNoting significant events that Noting significant events that affect, either positively or affect, either positively or negatively, the user’s ability to negatively, the user’s ability to accomplish a taskaccomplish a taskGather incidents over Gather incidents over participants, look for trends, participants, look for trends, patterns, correlationspatterns, correlations2121Questionnaires & Questionnaires & surveyssurveysDesign questions with analysis in Design questions with analysis in mindmind–Closed format more precise, easier to Closed format more precise, easier to analyzeanalyzeConvert qualitativeConvert qualitativequantitative measuresquantitative measuresYou give categories to usersYou give categories to users–Open-ended questions provide richer Open-ended questions provide richer feedback, longer to analyzefeedback, longer to analyzeUsers give you categoriesUsers give you categories2222Designing survey Designing survey questionsquestionsMultiple choiceMultiple choice–Collecting informationCollecting informationOrdinal rankingOrdinal ranking–Expressing relative preferencesExpressing relative preferencesLikert scalesLikert scales–Expressing personal reactionsExpressing personal reactions2323Likert scalesLikert scalesAsk users to rate on a numeric scaleAsk users to rate on a numeric scaleOdd number scale allows a neutral Odd number scale allows a neutral midpoint (5- or 7-point scale)midpoint (5- or 7-point scale)Even number scale forces taking a Even number scale forces taking a position (4- or 6-point scale)position (4- or 6-point scale)““Anchors” give examples of points Anchors” give examples of points along the scalealong the scale2424Semi-structured Semi-structured interviewsinterviewsInteractively asking questions (face-Interactively asking questions (face-to-face, telephone)to-face, telephone)Give users chance to explain “why” to Give users chance to explain “why” to complement “what” they did, complement “what” they did, subjective user’s viewpointsubjective user’s viewpointCan help with design questionsCan help with design questions–““What improvements would you What improvements would you suggest?”suggest?”Can be done individually or in groupsCan be done individually or in groups2525Quantitative measures Quantitative measures (comparison)(comparison)Independent variablesIndependent variables–Attributes we manipulate / vary in conditionAttributes we manipulate / vary in condition–Levels, value of attributeLevels, value of attributeDependent variablesDependent variables–Outcome of experiment, measures to Outcome of experiment, measures to evaluateevaluate–Usually measure user performanceUsually measure user performanceTime to completionTime to completionErrorsErrorsAmount of productionAmount of productionMeasures of satisfactionMeasures of satisfactionFlow path trackingFlow path tracking2626Experiment design (2)Experiment design (2)Control variablesControl variables–Attributes that remain the same across Attributes
View Full Document
Unlocking...