Model-Based Evaluation of Expert CellPhone Menu InteractionROBERT ST. AMANT and THOMAS E. HORTONNorth Carolina State UniversityandFRANK E. RITTERThe Pennsylvania State UniversityWe describe concepts to support the analysis of cell phone menu hierarchies, based on cognitivemodels of users and easy-to-use optimization techniques. We present an empirical study of userperformance on five simple tasks of menu traversal on an example cell phone. Two of the modelsapplied to these tasks, based on GOMS and ACT-R, give good predictions of behavior. We use theempirically supported models to create an effective evaluation and improvement process for menuhierarchies. Our work makes three main contributions: a novel and timely study of a new, verycommon HCI task; new versions of existing models for accurately predicting performance; and asearch procedure to generate menu hierarchies that reduce traversal time, in simulation studies,by about a third.Categories and Subject Descriptors: H.5.2 [Information Interfaces and Presentation]: UserInterfaces—Evaluation/methodologyGeneral Terms: Human Factors, DesignACM Reference Format:St. Amant, R., Horton, T. E., and Ritter, F. E. 2007. Model-based evaluation of expert cell phonemenu interaction. ACM Trans. Comput.-Hum. Interact. 14, 1, Article 1 (May 2007), 24 pages. DOI= 10.1145/1229855.1229856 http://doi.acm.org/10.1145/1229855.1229856This research was supported by the National Science Foundation (NSF) (IIS-0083281 and ITR-046852), the Space and Naval Warfare Systems Center, San Diego (N66001-1047-411F), and theOffice of Naval Research (N00014-02-1-0021, N00014-03-1-0248, and N00014-06-1-0164).The information in this article does not necessarily reflect the position or policies of the U.S. gov-ernment, and no official endorsement should be inferred.Authors’ addresses: R. St. Amant and T. E. Horton, Department of Computer Science, NorthCarolina State University, Raleigh, NC 27695; email:{stamant; horton}@csc.ncsu.edu; F. E. Ritter,College of Information Sciences and Technology, The Pennsylvania State University, UniversityPark, PA 16802; email: [email protected] to make digital or hard copies of part or all of this work for personal or classroom use isgranted without fee provided that copies are not made or distributed for profit or direct commercialadvantage and that copies show this notice on the first page or initial screen of a display alongwith the full citation. Copyrights for components of this work owned by others than ACM must behonored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers,to redistribute to lists, or to use any component of this work in other works requires prior specificpermission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 PennPlaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or [email protected]2007 ACM 1073-0616/2007/05-ART1 $5.00 DOI 10.1145/1229855.1229856 http://doi.acm.org/10.1145/1229855.1229856ACM Transactions on Computer-Human Interaction, Vol. 14, No. 1, Article 1, Publication date: May 2007.2•R. St. Amant et al.1. INTRODUCTIONThere are 2 billion cellular telephones in use today, and this number is expectedto reach 3 billion in 2008 [DiPrima 2006]. Cell phones are used for more thanmaking calls; they now include tools for managing contact information, voicemail, and hardware settings, and often software for playing games, browsingthe Web, and connecting to specialized information services. The market pen-etration of cell phones is much higher than that of conventional computers,which raises significant opportunities and challenges for HCI.This article presents techniques for evaluating and improving cell phoneusability, in particular the usability of the hierarchical menus that provideaccess to most functionality aside from dialing and data entry. While cell phonemenu interfaces may appear simple at first glance, they pose a nontrivial designproblem. Consider the menu hierarchy for the Kyocera 2325 cell phone, the first25 items of which are shown in Table I. If we count as terminals those selectionsthat open an application (e.g., a game), a list of data (e.g., recent calls), or aset of choices in the cell phone equivalent of a dialog box (e.g., for setting theringer volume), then this hierarchy contains 98 terminals, reachable through22 intermediate selections. The longest menu contains 12 items–all associatedwith the selection of different sounds. The shortest menu contains a single item,for entering a voice memo. Terminals in the hierarchy are up to four levels deep,and the mean number of actions to reach an item (scrolling plus selection), overall 98 terminals, is 13.3, taking on the order of 7 seconds for an experienceduser.This menu hierarchy is as large as that of a moderately sized desktop ap-plication (e.g., Eudora 5.2 with 103 items). This is not unusual for cell phones;the menu hierarchy for the Samsung MM-A800, which includes a digital cam-era, contains a remarkable 583 items [Pogue 2005]. Designing menu systemsfor any platform, including desktop systems, can be challenging, but for cellphones the problem is made more difficult by several factors:—Discrete selection actions in the form of button presses1are usually neededto move between menu items, because most cell phones lack more directselection capabilities (e.g., a mouse or touch screen).—Cell phone displays are small, allowing only a few menu items to be displayedat a single time. Many cell phones lack functionality for paging up or down,making display limitations even more significant.—There is less standardization in hardware supporting menu traversal for cellphones than for desktop machines. Some phones have two-way directionalbuttons, others four-way; some have a labeled “Menu” button, while othersrely on a button with overloaded functionality. Button placement can varysignificantly, with “Cancel” and “OK” buttons reversed from one phone toanother. If interfaces are developed for the lowest common denominator, in-dependently of specific hardware (which is common practice at the mobileapplication level), then even cell phones with elaborate interaction supportbecome less efficient.1We use the terms “button presses” and “key presses” interchangeably.ACM Transactions on Computer-Human Interaction, Vol. 14, No. 1, Article 1, Publication date: May 2007.Evaluation of Expert Cell Phone Menu