1Fall 2005 6.831 UI Design and Implementation 1 2Fall 2005 6.831 UI Design and Implementation 2 Source: Interface Hall of ShameIf you can’t read this, go to http://uid.csail.mit.edu/6.831/L16.gifOnce upon a time, this bizarre help message was popped up by a website (Midwest Microwave) when users requested to view the site’s product catalog. The message appears before the catalog is displayed. Clearly this message is a patch for usability problems in the catalog itself, but the patch itself has so many usability issues that it’s worth listing a few.•Overwhelming the user with detail. What’s important here, and what isn’t? (minimalist design)•Horrible layout: no paragraphs, no headings, no whitespace to guide the eye (aethestic design)•No attempt to organize the material into chunks so that it can be scanned, to find out what the user doesn’t already know (flexibility and efficiency)•This information is useless and out of context before the user has seen the task they’ll be faced with (help and documentation)•It’s a modal dialog box, so all this information will go away as soon as the user needs to get to the catalog (minimize memory load)•Using technical terms like V.90 modem (speak the user’s language)•“Please carefully jot down the Model Numbers” (recognition, not recall)•Poor response times: 20-60 second response times (user control and freedom), though in fairness this was common for the web at the time, and maybe Acrobat has sufficient progress interfaces to make up for it.•Misspelling “our catalog” in the first line (speak the user’s language, but really we don’t need a heuristic to justify pointing out spelling errors)3Fall 2005 6.831 UI Design and Implementation 3 User testing Ethics Formative evaluationIn this lecture, we’ll talk about user testing: putting an interface in front of real users. There are several kinds of user testing, but all of them by definition involve human beings, who are thinking, breathing individuals with rights and feelings. When we enlist the assistance of real people in interface testing, we take on some special responsibilities. So first we’ll talk about the ethics of user testing, which apply regardless of what kind of user test you’re doing.The rest of the lecture will focus on one particular kind of user test: formative evaluation, which is a user test performed during iterative design with the goal of finding usability problems to fix on the next design iteration.4Fall 2005 6.831 UI Design and Implementation 4 Formative evaluation Find problems for next iteration of design Evaluates prototype or implementation, in lab, on chosen tasks Qualitative observations (usability problems) Field study Find problems in context Evaluates working implementation, in real context, on real tasks Mostly qualitative observations Controlled experiment Tests a hypothesis (e.g., interface X is faster than interface Y) Evaluates working implementation, in controlled lab environment,on chosen tasks Mostly quantitative observations (time, error rate, satisfaction)Here are three common kinds of user tests.You’ve already done a formative evaluation with your paper prototypes. The purpose of formative evaluation is finding usability problems in order to fix them in the next design iteration. Formative evaluation doesn’t need a full working implementation, but can be done on a variety of prototypes. This kind of user test is usually done in an environment that’s under your control, like an office or a usability lab. You also choose the tasks given to users, which are generally realistic (drawn from task analysis, which is based on observation) but nevertheless fake. The results of formative evaluation are largely qualitative observations, usually a list of usability problems.Note that Prototype Testing Day was not the best way to do formative evaluation: first, because your classmates are probably not representative of your target user population; and second, because we had artificial time constraints that raised the pressure on users and experimenters, prevented using substantial tasks, and didn’t allow for much debriefing or discussion after the test. Better user tests would use representative users and be more relaxed (unless time pressure is a key attribute of the actual working context of your interface, in which case maybe you’d want to try to simulate those conditions intentionally).A key problem with formative evaluation is that you have to control too much. Running a test in a lab environment on tasks of your invention may not tell you enough about how well your interface will work in a real context on real tasks. A field study can answer these questions, by actually deploying a working implementation to real users, and then going out to the users’ real environment and observing how they use it. We won’t say much about field studies in this class.A third kind of user test is a controlled experiment, whose goal is to test a quantifiable hypothesis about one or more interfaces. Controlled experiments happen under carefully controlled conditions using carefully-designed tasks – often more carefully chosen than formative evaluation tasks. Hypotheses can only be tested by quantitative measurements of usability, like time elapsed, number of errors, or subjective ratings. We’ll talk about how to design controlled experiments in a later lecture.5Fall 2005 6.831 UI Design and Implementation 5! Users are human beings Human subjects have been seriously abused in the past Nazi concentration camps Tuskegee syphilis study MIT Fernald School study: feeding radioactive isotopes to mentally retarded children Yale electric shock study Research involving user testing is now subject to close scrutiny MIT Committee on Use of Humans as Experimental Subjects (COUHES) must approve research-related user studiesLet’s start by talking about some issues that are relevant to all kinds of user testing: ethics. Human subjects have been horribly abused in the name of science over the past century. Here are some of the most egregious
View Full Document
Unlocking...