1Fall 2004 6.831 UI Design and Implementation 1 2Fall 2004 6.831 UI Design and Implementation 2 Today’s candidate for the Hall of Fame or Shame is adaptive menus, a feature of Microsoft Office. Initially, a menu shows only the most commonly used commands. Clicking on the arrow at the bottom of the menu expands it to show all available commands. Adaptive menus track how often a user invokes each command, in order to display frequently-used commands and recently-used commands.This interface is striving for a compromise between simplicity (i.e., providing as few features as possible) and task analysis (supporting the tasks required by many users, and trying to adapt to the common tasks of each individual user). Both properties are important for usability. Unfortunately they also compete with each other. Adaptive menus are an interesting hybrid technique that’s trying to satisfy both.The downside is lack of predictability. Because the menu may change in complex and unpredictable ways – with new items appearing and underused items disappearing for no visible reason – the user can no longer rely on a lot of useful cues to find menu items. One of these cues that’s lost is spatial memory – Paste may be found at different distances down the menu each time the menu appears. Another missing cue is context: Paste’s neighbors may change as well.Another downside is lack of user control. The adaptation happens automatically; the user can’t manually fixate or remove items from a menu.This particular implementation of adaptive menus has some specific usability problems. When the full menu appears, the new items are inserted into place, and there’s very little contrast in the graphic design to distinguish between the old items and the new items. So the user has to search through the entire menu again.But this particular implementation addresses other usability problems very well. When the user is hunting through all the menus, looking for a command, the full menu only has to be requested once; then all subsequent menus are fully displayed.3Fall 2004 6.831 UI Design and Implementation 3 !" Experiment designToday’s lecture covers some of the issues involved in designing a controlled experiment. The issues are general to all scientific experiments, but we’ll look specifically at how they apply to user interface testing.4Fall 2004 6.831 UI Design and Implementation 4# Start with a testable hypothesis Interface X is faster than interface Y Manipulate independent variables different interfaces, user classes, tasks Measure dependent variables times, errors, satisfaction Use statistical tests to accept or reject the hypothesisHere’s a high-level overview of a controlled experiment. You start by stating a clear, testablehypothesis. By testable, we mean that the hypothesis must be quantifiable and measurable. For example, your hypothesis might be, “menu bars are faster than a Gimp-style right-click menu with hierarchical submenus”. Here’s another example that we’ll use throughout this lecture: suppose you’ve developed two materials for the soles of children’s shoes. Then your hypothesis might be, “material A wears slower than material B.”You then choose your independent variables – the variables you’re going to manipulate in order to test the hypothesis. In our example, the independent variable is the kind of interface, menubar or right-click menu. Other independent variables may also be useful. For example, you may want to test your hypothesis on different user classes (novices and experts, or Windows users and Mac users). You may also want to test it on certain kinds of tasks. For example, in one kind of task, the menus might have an alphabetized list of names; in another, they might have functionally-grouped commands. In the shoe sole example, the independent variable would be the type of material used to make the shoe sole.You also have to choose the dependent variables, the variables you’ll actually measure in the experiment to test the hypothesis. Typical dependent variables in user testing are time, error rate, event count (for events other than errors – e.g., how many times the user used a particular command), and subjective satisfaction (usually measured by a questionnaire). In the shoe example, the dependent variable might be the thickness of the sole after a subject has worn it for a while.Finally, you use statistical techniques to analyze how your changes in the independent variables affected the dependent variables, and whether those effects are significant (indicating a genuine cause-and-effect) or not (merely the result of chance or noise). We’ll say a little more about statistical tests in the next lecture.5Fall 2004 6.831 UI Design and Implementation 5 " $ % & Hypothesis: Mac menu bar is faster than Windows menu bar Independent: position of menu bar Dependent: time to reach menu barHere’s an example of a hypothesis that we might want to test: that the Macintosh menu bar, which is anchored to the top of the screen, is faster to access than the Windows menu bar, which is separated from the top of the screen by a window title bar.The independent variable here is the position of the menu bar: either y = 0 or y = 16 (or whatever the height of the title bar is).The dependent variable we might measure is time: how long it takes the user to move the mouse up to the menu bar and click on a particular target menu to pull it down.6Fall 2004 6.831 UI Design and Implementation 6 '$ Processindependentvariablesdependentvariablesunknown/uncontrolledvariablesXYY = f(X) + Here’s a block diagram to help you visualize what we’re trying to do with experiment design. Think of the process you’re trying to understand (e.g., menu selection) as a black box, with lots of inputs and a few outputs. A controlled experiment twiddles some of the input knobs on this box (the independent variables) and observes some of the outputs (the dependent variables) to see how they are
View Full Document
Unlocking...