Spring 2008 6.831 User Interface Design and Implementation 1Lecture 13: User Testing UI Hall of Fame or Shame?Spring 2008 6.831 User Interface Design and Implementation 2Suggested by Ryan Damico Today’s candidate for the Hall of Shame is this entry form from the 1800Flowers web site. The purpose of the form is to enter a message for a greeting card that will accompany a delivered flower arrangement. (So you can see the whole interface, I’ve moved the Greeting Type drop-down menu to the right. In the real interface, it appears where you’d expect, right under the Greeting Type drop-down box.) How does this interface fare with respect to: - user control and freedom - visibility - consistency - speaking the user’s languageToday’s Topics• User testing•Ethics• Formative evaluationSpring 2008 6.831 User Interface Design and Implementation 3 In this lecture, we’ll talk about user testing: putting an interface in front of real users. There are several kinds of user testing, but all of them by definition involve human beings, who are thinking, breathing individuals with rights and feelings. When we enlist the assistance of real people in interface testing, we take on some special responsibilities. So first we’ll talk about the ethics of user testing, which apply regardless of what kind of user test you’re doing. The rest of the lecture will focus on one particular kind of user test: formative evaluation, which is a user test performed during iterative design with the goal of finding usability problems to fix on the next design iteration. Kinds of User Tests• Formative evaluation– Find problems for next iteration of design– Evaluates prototype or implementation, in lab, on chosen tasks– Qualitative observations (usability problems)• Field study– Find problems in context– Evaluates working implementation, in real context, on real tasks– Mostly qualitative observations• Controlled experiment– Tests a hypothesis (e.g., interface X is faster than interface Y)– Evaluates working implementation, in controlled lab environment, on chosen tasks– Mostly quantitative observations (time, error rate, satisfaction)Spring 2008 6.831 User Interface Design and Implementation 4 Here are three common kinds of user tests. You’ll be doing a formative evaluation with your paper prototypes. The purpose of formative evaluation is finding usability problems in order to fix them in the next design iteration. Formative evaluation doesn’t need a full working implementation, but can be done on a variety of prototypes. This kind of user test is usually done in an environment that’s under your control, like an office or a usability lab. You also choose the tasks given to users, which are generally realistic (drawn from task analysis, which is based on observation) but nevertheless fake. The results of formative evaluation are largely qualitative observations, usually a list of usability problems. Note that Prototype Testing Day is not the best way to do formative evaluation: first, because your classmates are probably not representative of your target user population; and second, because we have artificial time constraints that raised the pressure on users and experimenters, prevent using substantial tasks, and don’t allow for much debriefing or discussion after the test. Better user tests would use representative users and be more relaxed (unless time pressure is a key attribute of the actual working context of your interface, in which case maybe you’d want to try to simulate those conditions intentionally).A key problem with formative evaluation is that you have to control too much. Running a test in a lab environment on tasks of your invention may not tell you enough about how well your interface will work in a real context on real tasks. A field study can answer these questions, by actually deploying a working implementation to real users, and then going out to the users’ real environment and observing how they use it. We won’t say much about field studies in this class. A third kind of user test is a controlled experiment, whose goal is to test a quantifiable hypothesis about one or more interfaces. Controlled experiments happen under carefully controlled conditions using carefully-designed tasks – often more carefully chosen than formative evaluation tasks. Hypotheses can only be tested by quantitative measurements of usability, like time elapsed, number of errors, or subjective ratings. We’ll talk about how to design controlled experiments in a later lecture. Ethics of User Testing• Users are human beings– Human subjects have been seriously abused in the past• Nazi concentration camps• Tuskegee syphilis study• MIT Fernald School study: feeding radioactive isotopes to mentally retarded children• Yale electric shock study– Research involving user testing is now subject to close scrutiny• MIT Committee on Use of Humans as Experimental Subjects (COUHES) must approve research-related user studiesSpring 2008 6.831 User Interface Design and Implementation 5 Let’s start by talking about some issues that are relevant to all kinds of user testing: ethics. Human subjects have been horribly abused in the name of science over the past century. Here are some of the most egregious cases: In Nazi concentration camps (1940-1945), doctors used prisoners of war, political prisoners, and Jews as human guinea pigs for horrific experiments. Some experiments tested the limits of human endurance in extreme cold, low pressures, or exposure. Other experiments intentionally infected people with massive doses of pathogens, such as typhus; others tested new chemical weapons or new medical procedures. Thousands of people were killed by these experiments; they were criminal, on a massive scale. In the Tuskegee Institute syphilis study (1932-1972), the US government studied the effects of untreated syphilis in black men in the rural South. In exchange for their participation in the study, the men were given free health examinations. But they weren’t told that they had syphilis, or that the disease was potentially fatal. Nor were they giventreatment for the disease, even as proven, effective treatments like penicillin became available. Out of 339 men studied, 28 died directly of syphilis, 100 of related complications. 40 wives were infected, and 19 children were born with congenital syphilis. In the 1940s and 1950s, MIT researchers
View Full Document
Unlocking...