Pen-Based Interaction Techniques For Organizing Material on an Electronic Whiteboard Thomas R Moran, Patrick Chiu,* William van Melle Xerox Palo Alto Research Center, 3333 Coyote Hill Road. Palo Alto, CA 94304 (moran,vanrneIle)@parc.xerox.com, [email protected] ABSTRACT This paper presents a scheme for extending an informal, pen- based whiteboard system (the Tivoli application on- the Xerox LiveBoard) to provide interaction techniques that enable groups of users in informal meetings to easily orga- nize and.rearrange material and to manage the space on the board. The techniques are based on the direct manipulation of boundaries and the implicit recognition of regions. The techniques include operations for shrinking and rearranging, structured borders that tessellate the board, freeform enclo- sures that can be split, fused, and linked, and collapsible annotations. Experience with using these techniques, the results of a user test, some design trade-offs and lessons, and future directions are discussed. KEYWORDS: whiteboard metaphor, pen-based systems, freeform interaction, implicit structure, emergent structure, structural group- ing, informal systems, recognition-based systems, list structures, meeting support tools, gestural interfaces, user interface design 7. INTRODUCTION Our goal is to provide computational support for small, real- time “working meetings” in which groups interact to gener- ate and organize ideas (creating and assessing new ideas and perspectives, discussing them, organizing them, negotiating about them, and so on). Group interaction in such situations is informal, freewheeling, and fluid. Groups often use whiteboards to provide a shared visual sur- face to record the ideas they are working with and to pre- serve a shared context. When manipulation demands are high, people sometimes use a tackboard, magnetic board or tape and paper in order to be able to move items around. Sometimes people use projected computer applications, such as word processors or spreadsheets, to gain the advantages of both editing and saving the created material. However, these kinds of applications are not very well suited to meeting situ- ations, because they force users to create and work with for- malized representations. The overhead of using such representations inhibits the very processes they are meant to support [13]. One of the big challenges for current HCI PcmkGxl IO make digitnl/lwd copies of all or pa11 of this mnrerinl for pcrsotlnl or clnwoom use is gmnnlcd willlout fee provided 11x11 Ihe copies arc no1 mndr or distrilwlcd for prolit or commercial ndvantny, the copy- riglIt notice, tl~c litIt oftllc publication and its dale appear, and notice is giwn lllnt col~yrigllt is by pt2nGrsion oftllc ACM, Inc. TO Copy otlicwise, to republisl~, to posl 011 SWWIX or IO rcdislributc IO lisls. requires specific pcnnission and/or fru UIS’T’ 97 BnnJJ Alhertn, Canada Copyright I997 ACM O-89791-8X1-9/37/lO..S3.50 design is to create systems to support informal interaction. Pen-based systems that allow scribbling on wall-size dis- plays or notepads can support whiteboard or shared note- book metaphors for working with informally scribbled material [4]. The free, easy, and familiar expression permit- ted by such systems makes them a promising class of tools to support informal interaction in meetings [15]. We are explor- ing pen-based techniques usable by people without any spe- cial drawing skills, i.e., most people who go to meetings. We have been working for several years on a program of research to provide computational meeting tools based on a whiteboard metaphor. Our idea is to allow freehand/freeform creation and manipulation of materials on the whiteboard and to provide facilities to easily and noncommittally orga- nize and structure the materials as needed by the group. What makes this program of research possible is the Live- Board [3], a large, shared, pen-based, rear-projected, elec- tronic display. This provides a whiteboard-size interactive display that allows us to experience and experiment with “group-computer interaction” in a way not possible with smaller displays, such as workstations, or non- or semi-inter- active, front-projected displays. We have developed a software application, called Tivoli [ll], that simulates whiteboard functionality on the LiveBoard.’ Tivoli provides basic pen-based scribbling and erasing inter- action, plus facilities for editing materials by pen-based ges- turing and wiping techniques. We extended Tivoli to allow “implicit structuring” [9] of the material on the board.* Material is created on the board in a 1. Tivoli is a research prototype used at PARC [8]. It is written in C++ and runs under Unix and X Windows. The LiveBoard is a PC-based commercial product from Liveworks, Inc. Tivoli is run on the LiveBoard through an X server on the LiveBoard. (A PC-based commercial product from Liveworks, called MeetingBoard, is an extended early version of Tivoli.) 2. We use the term “board” in this paper to refer to the visible work surface provided by Tivoli on the LiveBoard display. * Current address: Patrick Chiu, Fuji Xerox, Palo Alto Laboratory, 3400 Hillview Avenue, Palo Alto, CA 94304. At the time of the work reported in this paper, Patrick Chiu was with Liveworks, Inc.. A Xerox Company, working at Xerox PARC. 45freeform manner, which means that anything can go any- where without constraint. However, the user can indicate to the system by certain gestures that the material is to be regarded temporarily as having a certain structure, such as handwritten text. Tivoli will then apply editing operations in accordance with the conventions of that structure, such as opening and closing space where needed when moving words around (similar capabilities are provided by [ 13). We have now further extended T&S by developing a set of simple and natural techniques for helping users spatially organize material on the board. This paper presents, in Sec- tion 2, the motivation, approaches, and design principles for these new techniques. Section 3 describes the techniques themselves. Finally, in