User-Centered Design and DevelopmentCopyright Notice484-W09 QuarterChapter 13 Asking Users and ExpertsChapter OverviewMotivationObjectivesAsking users & expertsThe aimsInterviewsBasics of interviewingThings to avoid when preparing interview questionsComponents of an InterviewInterview ProcessProbes and PromptsGroup InterviewsAnalyzing Interview DataQuestionnairesQuestionnaire StyleLikert ScalesDeveloping a QuestionnaireEncouraging a good responseAdvantages of online questionnairesProblems with online questionnairesQuestionnaire Data Analysis and PresentationActivity: CSC Alumni QuestionnairesSUMIAsking ExpertsHeuristic EvaluationNielsen’s HeuristicsDiscount EvaluationStages Heuristic EvaluationAdvantages and problemsActivity: Heuristic Evaluation of CSC QuestionnairesCognitive walkthroughsCognitive Walkthrough QuestionsPluralistic WalkthroughKey pointsA project for you …Slide 41Slide 42User-Centered Design and Development Instructor: Franz J. KurfessComputer Science Dept.Cal Poly San Luis ObispoFJK 2005Copyright Notice•These slides are a revised version of the originals provided with the book “Interaction Design” by Jennifer Preece, Yvonne Rogers, and Helen Sharp, Wiley, 2002.•I added some material, made some minor modifications, and created a custom show to select a subset.–Slides added or modified by me are marked with my initials (FJK), unless I forgot it …FJK 2005484-W09 Quarter•The slides I use in class are in the Custom Show “484-W09”. It is a subset of the whole collection in this file.•Week 7 contains slides from Chapters 12 and 13 of the textbook. FJK 2005Chapter 13 Asking Users and ExpertsFJK 2005Chapter Overview•Asking Users–Interviews–Questionnaires•Asking Experts–Inspections–WalkthroughsFJK 2005FJK 2005Motivation•often it is easier to talk with users than to observe them•users can present their case, instead of using interpretations and guesses derived from observations•it can be better to talk to experts about the expected behavior of usersFJK 2005Objectives•be familiar with the main methods of asking users and experts•be aware of the differences between various interviewing techniques and types of questionnaires•know how to develop basic interviewing strategies and how to design questionnaires•know when to employ which method•gain experience in obtaining information from users and expertsAsking users & expertsThe aims–Discuss the role of interviews & questionnaires in evaluation.–Teach basic questionnaire design. –Describe how do interviews, heuristic evaluation & walkthroughs. –Describe how to collect, analyze & present data. –Discuss strengths & limitations of these techniquesInterviews•unstructured–not directed by a script–rich but not replicable•structured–tightly scripted, often like a questionnaire–replicable but may lack richness•semi-structured –guided by a script –interesting issues can be explored in more depth–can provide a good balance between richness and replicabilityBasics of interviewing•DECIDE framework can offer guidance•goals and questions –guide all interviews•types of questions–‘closed questions’ have a predetermined answer format, e.g., ‘yes’ or ‘no’•quicker to administer•easier to analyze•may miss valuable information –‘open questions’ do not have a predetermined format•may yield valuable information•may take longer•often more difficult to analyzeThings to avoid when preparing interview questions•long questions•compound sentences–split into two•jargon–language that the interviewee may not understand •leading questions–make assumptions •e.g., why do/don’t you like …?•unconscious biases–e.g. gender, age stereotypesComponents of an Interview•introduction –introduce yourself, explain the goals of the interview, reassure about the ethical issues, ask to record, present an informed consent form.•warm-up–make first questions easy and non-threatening. •main body–present questions in a logical order•a cool-off period –include a few easy questions to defuse tension at the end•closure–thank interviewee, signal the end (switch recorder off)Interview Process•use the DECIDE framework for guidance•dress in a similar way to participants•check recording equipment in advance•devise a system for coding names of participants to preserve confidentiality•be pleasant•ask participants to complete an informed consent formProbes and Prompts•probes –devices for getting more information–e.g., ‘Would you like to add anything?’•prompts –devices to help interviewee•e.g. help with remembering a name•probing and prompting should not create bias•too much information can encourage participants to try to guess the answer–or the answer they think you would like to hearGroup Interviews•also known as ‘focus groups’•typically 3-10 participants•can provide a diverse range of opinions•need to be managed–ensure everyone contributes–discussion isn’t dominated by one person–the agenda of topics is covered–stay focusedAnalyzing Interview Data•depends on the type of interview•structured interviews –can be analyzed like questionnaires•unstructured interviews –generate data like that from participant observation–analyze as soon as possible to identify topics and themes from the dataQuestionnaires•questions can be closed or open–closed questions are easiest to analyze, and may be done by computer•can be administered to large populations–paper, email & the web used for dissemination•electronic questionnaires –data goes into a data base and is easy to analyze–requires computer skills, access•online questionnaires–sampling can be a problem when the size of a population is unknown–may not be representative of the whole group of usersQuestionnaire Style•varies according to goal –use the DECIDE framework for guidance•questionnaire format can include:–‘yes’, ‘no’ checkboxes–checkboxes that offer many options (radio buttons)–Likert rating scales–semantic scales–open-ended responsesLikert Scales •have a range of points–3, 5, 7 & 9 point scales are common–sometimes verbal descriptors are used•debate about which is bestDeveloping a Questionnaire•Provide a clear statement of purpose & guarantee participants anonymity •Plan questions–if developing a web-based questionnaire, design off-line first•Decide on whether