Usability EvaluationDr. Yan LiuDepartment of Biomedical, Industrial and Human Factors EngineeringWright State University2Introduction What is Usability Evaluation Assess the extent to which the product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use Usability Evaluation in Design Process Should occur throughout the design life cycle, with the results of the evaluation feeding back into modification to the design Usability Evaluation Methods Based on expert evaluation, without direct user involvement Particularly useful for assessing early designs and prototypes Involve users to study actual use of the system  Usually require a working prototype or implementation Users may also be involved in assessing early design ideas e.g. focus groups in which a group people are asked about their opinions of the system3Goals of Usability Evaluation Assess System’s Functionality The system’s functionality must accord with the user’s requirements Making the appropriate functionality available within the system Making the functionality clearly reachable by the user in terms of the actions that the user needs to take to perform the tasks Assess Users’ Experience with the Interaction Aspects such as how easy the system is to learn and use and the users’ satisfaction with it Users’ enjoyment and emotional response (particularly in systems aimed at entertainment) Identify Specific Problems with the Design Aspects of the design which, when used in their intended context, cause unexpected results or confusion among users4Evaluation Though Expert Analysis Overview The basic intention is to identify any areas that are likely to cause difficulties because they violate known design rules or ignore accepted empirical results Flexible and can be used at any stage in the development process  Design specification, storyboards and prototypes, full implementations Relatively cheap Do not assess actual use of the system Approaches Cognitive walkthrough Heuristic evaluation Use of models Use of previous work5Cognitive Walkthrough Overview Proposed in Polson et al. (1992) Main focus is usually to establish how easy a system is to support exploratory learning Phase One  Collect Information about the System, Users, and Tasks A fairly detailed specification or prototype of the system An indication of who the users are and what kind of experience and knowledge the evaluators can assume about them A description of the representative tasks most users would want to perform on the system A complete, written list of actions needed to complete the tasks with the proposed system6Cognitive Walkthrough (Cont’d) Phase Two  The evaluators step through the action sequence identified earlier to critique the system and tell a believable story about its usability by asking themselves a set of questions for each step Q1: Is the effect of the action the same as the user’s goal at that point? e.g. If the effect of an action is to save a document, is “saving a document” what the user wants to do? Q2: Will the user see that the action is available? (visibility of the action) e.g. Will the user see the control that is used to save a document? Q3: Once the user has found the correct action, will he/she know it is the one he/she needs? (meaning and effect of the action) e.g. Even if the user can see the control, will the user recognize that it is the one he/she is looking for to complete the task? Q4: After the action is taken, will the user understand the feedback he/she gets? Appropriate feedback should be provided to inform the user of what has happened7Cognitive Walkthrough (Cont’d) Phase Two (Cont.) Document the cognitive walkthrough to keep a record of the evaluators’ evaluation results Pros and cons of the system It is a good idea to produce some standard evaluation forms for the walkthrough The cover form would list the four questions asked during the walkthrough process, as well as the dates and time of the walkthrough and the names of the evaluators For each action, a separate standard form is filled out that answers each of the four questions Any negative answer for a particular action should be documented in detail on a separate usability problem report sheet, including the severity of the problem8Time: 21:45Channel: 312 345 678 90The Handset in Normal Use Suppose we are designing a remote control for a video recorder (VCR) and interested in the task of programming the VCR to do timed recordings. Our initial design is shown in the following figures. This VCR allows the user to program up to three timed recordings in different “streams”. The next available stream number is automatically assigned. We want to evaluate the design using the cognitive walkthrough method. 12 345 678 90Start: | End:Channel: Date: After the “Time-Record” Button Has Been Pressed912 345 678 90Start: | End:Channel: Date: After the “Time-Record” Button Has Been Pressed Collect information about the system, users, and tasks • We can assume that the user is familiar with VCRs but not with this particular design • Identify a representative task: programming the video to time record a program starting at 18:00 and finishing at 19:15 on channel 4 on Oct. 16, 2009• Specify the action sequence for the task in terms of the user’s action (UA) and the system’s display or response (SD)• UA1: Press the “time-record” button• SD1: Display moves to timer mode. Flashing cursor appears after “Start:”• UA2: Press digits 1 8 0 0• SD2: Each digit is displayed as typed and flashing cursor moves to next position• UA3: Press the “time-record” button• SD3: Flashing cursor moves to after “End:”• UA4: Press digits 1 9 1 5• SD4: Each digit is displayed as typed and flashing cursor moves to next position• UA5: Press the “time-record” button• SD5: Flashing cursor moves to after “Channel:”• US6: Press digit 4• SD6: Digit is displayed as typed and flashing cursor moves to next position• UA7: Press the “time-record” button• SD7: Flashing cursor moves to after “Date:”• US8: Press digits 16 10 0 8• SD8: Each digit is displayed as typed and flashing cursor moves to next position• UA9: Press the