1 Cover Page Title of submission: Active Capture Design Case Study: SIMS Faces Category of submission: Design Case Study Name and full contact address (surface, fax, email) of the individual responsible for submitting and receiving inquiries about the submission: Ana Ramírez Chang, University of California at Berkeley, 360 Hearst Mining, Berkeley, CA 94720, USA, +1 510.847.1985, [email protected] Active Capture Design Case Study: SIMS Faces Abstract We present a design case study for the SIMS Faces application. The SIMS Faces application is an Active Capture [1] application that works with the user to take her picture and record her saying her name for inclusion on the department web page. Active Capture applications are systems that capture and direct human action by working with the user, directing her and monitoring her progress, to complete a common goal, in this case taking her picture when she is smiling and looking at the camera. In addition to producing a working Active Capture application, the project also included studying the design of Active Capture applications. The team conducted an ethnographic study [2] to inform the design of the interaction with the user, prototyped a set of tools to support the design process, and iterated a design process involving bodystorming, a Wizard-of-Oz study, the prototyped tools, and a user test of the implemented application. Keywords Interaction Design, Interdisciplinary Design, Prototyping, System Design, User-Centered Design / Human-Centered Design, User Experience, User Interface Design, Audio, Video, Vision, Visualization. Project/problem statement We developed the SIMS Faces application, an Active Capture [1] application that works with the user to record her saying her name, and take her picture for inclusion on the department web page. The application allows the department staff to easily capture a picture Ana Ramírez Chang Garage Cinema Research Berkeley Institute of Design Computer Science Division University of California at Berkeley Berkeley, CA 94720 [email protected] Marc Davis Garage Cinema Research School of Information Management and Systems University of California at Berkeley Berkeley, CA 94720 [email protected] Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright © 2005 AIGA | The professional association for design.3 of each new student and an audio clip of each student pronouncing her name without having to take the pictures and record the names manually. The new students can use the application as many times as they would like without placing a burden on the department staff. The application was tested over the spring semester 2004 and was deployed for incoming students for the fall 2005 semester. In developing the SIMS Faces application, we had two main goals: 1) design and implement an application to be deployed in the University of California at Berkeley, School of Information Management and Systems (SIMS); and 2) study and improve the design process for Active Capture applications. In this paper we narrate the design and implementation of the SIMS Faces application and present subprojects that emerged during the project to study and support the design of Active Capture applications. These subprojects include a set of contextual interviews to inform the design of the interaction with the user, and a set of tools to support the design process. Background ACTIVE CAPTURE ”Active Capture” is a new paradigm in multimedia computing and applications that brings together capture, interaction, and processing and exists in the intersection of these three capabilities (See Figure 1). Most current human-computer interfaces largely exclude media capture and exist at the intersection of interaction and processing. In order to incorporate media capture into an interaction without requiring signal processing that would be beyond current capabilities, the interaction must be designed to leverage context from the interaction. For example, if the system wants to take a picture of the user smiling, it can interact with the user to get them to face the camera and smile and use simple, robust parsers (such as an eye finder and mouth motion detector) to aid in the contextualized capture, interaction, and processing. In this paper, we will refer to two Active Capture applications, the SIMS Faces application (the development of which this paper narrates) and the Kiosk Demo. Both systems are shown in a video at: www.cs.berkeley.edu/~anar/chang_simsfaces_CASE.mpg The SIMS Faces application works with the user to achieve two goals, take her picture, and record her saying her name. The Kiosk Demo is similar to a picture kiosk in the mall, but instead of taking the user’s picture, it takes a few videos of the user, and automatically creates a personalized commercial or Capture Active Capture Interaction Processing Direction/ Cinematography Human Computer Interaction Computer Vision/ AuditionFigure 1. Active Capture brings together capture, interaction and processing and exists in the intersection of these three capabilities. Figure 2. Pictures of an Active Capture participant performing a head turn. The figure shows both the original captured footage and corresponding images from an automatically generated Terminator IItrailer. See a video of the system at: www.cs.berkeley.edu/~anar/chang_simsfaces_CASE.mpg Before head turn After head turn4 movie trailer staring the user. There are two parts in the Kiosk Demo, the Active Capture part works with the user to capture a shot of her looking at the camera and screaming, and a shot of her turning her head to look at the camera. The second part of the Kiosk Demo uses Adaptive Media technology described in [3, 4]. The shots of the user screaming and turning her head are automatically edited into a variety of commercials and movie trailers including a 7up commercial, an MCI commercial, and the Terminator II movie trailer. Figure 2 shows how the head turn is parsed into the Terminator II movie trailer. COMPONENTS OF ACTIVE CAPTURE APPLICATIONS Active Capture applications are made of