Video-as-data and digital video manipulation techniques for transforming learning sciences research, education and other cultural practices* Roy D. Pea Stanford University “Only in the stream of thought and life do words have meaning.” Ludwig Wittgenstein (1981) Zettel, (2nd. Ed.) G.E.M. Anscombe and G.H.V. Wright (Eds.). Oxford: Blackwell. (No. 173) Abbreviated title for running headline: Video-as-data (To appear in J. Weiss, J. Nolan & P. Trifonas (Eds.), International handbook of virtual learning environments. Dordrecht: Kluwer Academic Publishing, 2005. Please do not quote without permission.) Stanford Center for Innovations in Learning (SCIL) Wallenberg Hall, Building 160 Stanford, CA 94305 Phone: 650.724.3720 Fax: 650.724.0432 Email: [email protected] * DIVER™, WebDIVER™, Dive™ and “Guided Noticing”™ are trademarks of Stanford University for DIVER software and affiliated services with patents pending. The DIVER project work has been supported by grants from the National Science Foundation (#0216334, #0234456, #0326497, #0354453) and the Hewlett Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of these funders. Roy Pea served as lead author with contributing authors Ken Dauber, Michael Mills, Joseph Rosen, Wolfgang Effelsberg and Eric Hoffert for portions of earlier versions of some of the text material provided here regarding DIVER. Ray Pacheone, Randall Souviney and Peter Youngs have been instrumental in our thinking of DIVER in the teacher preparation and certification context. Brian MacWhinney was an additional key collaborator in conceptualizing the Digital Video Collaboratory characterized in Section 8.0.Roy Pea Video as Data (Draft 12/30/2004) 2 1. Introduction This chapter concerns the theoretical and empirical foundations and current progress of the DIVER Project at Stanford University. The DIVER Project—Digital Interactive Video Exploration and Reflection—aspires to accelerate cultural appropriation of video as a fluid expressive medium for generating, sharing and critiquing different perspectives on the same richly recorded events and to work with others to establish a Digital Video Collaboratory that enables cumulative knowledge building from video-as-data for discovery and commentary. These uses of digital video manipulation are very distinctive from those used in virtual learning environments today across K-12, higher education and corporate training (e.g., BlackBoard, WebCT, PlaceWare), which are primarily video clips that are used to illustrate a point or concept during a lecture or a video of a faculty member teaching and using PowerPoint slides. The DIVER system distinctively enables “point of view” authoring of video tours of archival materials (from video to animations and static imagery) in a manner that supports sharing, collaboration, and knowledge building around a common ground of reference. A fundamental goal is user-driven content re-use, prompted by the desire of content users to reinterpret content in new ways, and to communicate and share their interpretations with others, for purposes ranging from e-learning to entertainment. DIVER makes it possible to easily create an infinite variety of new digital video clips from a video record. A user of DIVER software "dives" into a video record by input controlling—with a mouse, joystick, or other input device—a virtual camera that can zoom and pan through space and time within an overview window of the source video. The virtual camera dynamically crops still image clips, or records multi-frame video “pathways” through normal consumer 4:3 aspect ratio video or a range of parameters (e.g., 20:3) for video records captured with a panoramic camera1, to create a dive™ (a DIVER worksheet). A dive is made up of a collection of re-orderable “panels”, each of which contains a small key video frame (often called a “thumbnail”) representing a clip as well as a text field that may contain an accompanying annotation, code, or other interpretation. After creating a dive using the desktop DIVER application, the user can upload it onto WebDIVER™, a Website for interactive browsing, searching, and display of video clips and collaborative commentary on dives. In an alternative implementation, one can dive on streaming video files that are made accessible through a web server over the Internet, without either requiring the downloading of a DIVER desktop application or the media files upon which the user dives. In the desktop DIVER implementation, the dive itself is packaged automatically as an XML document with associated media files. XML (Extensible Markup Language) is the universal language approved in 1998 by the World Wide Web Consortium (W3C) for structured documents and data on the web. The Web representation of the dive includes 1 A panoramic video camera is created when one or more video cameras are combined with one or more mirrors to capture 360-degree horizontal data around the panoramic camera’s fixed-point location. Image processing software is then used to “dewarp” the imagery that is produced from the reflective surface and to create interpretable displays on a computer monitor of the captured video data. A common method for depicting the panoramic video record is a wide-aspect ratio window that looks as if the cylinder has been sliced and “peeled-back” (see Figure 2 for illustration).Roy Pea Video as Data (Draft 12/30/2004) 3 the key frames, video clips and annotations. A dive can be shared with colleagues over the Internet and become the focus of knowledge building, argumentative, tutorial, assessment or general communicative exchanges. A schematic representation of the recording, “diving” and Web-sharing phases is shown in Figure 1: Figure 1. Overview of the DIVER video exploration and reflection system Much of our primary DIVER work involves scenarios in which a video camera is used to record complex human interactions such as the behavior of learners and teachers in a classroom, or research group meetings. One may capture video directly into DIVER using DIVER’s MovieMaker feature with a digital videocamera connected by FireWire to the computer, or use DIVER’s MovieImporter feature to bring in as