© SCS REAL-TIME CAMERA CONTROL FOR INTERACTIVE STORYTELLING Fred Charles, Jean-Luc Lugrin, Marc Cavazza and Steven J. Mead School of Computing and Mathematics University of Teesside Middlesbrough, TS1 3BA, United Kingdom E-mail: {f.charles, j.l.lugrin, m.o.cavazza, steven.j.mead}@tees.ac.uk KEYWORDS Automatic Camera Control, Interactive Storytelling, AI-based Animation, Computer Games. ABSTRACT In this paper, we present a fully implemented prototype of real-time cinematic control for character-based interactive storytelling approaches, where story diversity emerges from dynamic interaction between characters. We describe the specifities of real-time cinematic control within a dynamic virtual environment, where events occur at different locations at the same time. We also present results based on a situation of interaction between characters in the unfolding of a story. INTRODUCTION Our interactive storytelling system (Cavazza et al., 2002) is based on the interaction between characters’ behaviours during the unfolding of the main characters’ plans representing their role-play. The (active) spectator can intervene with situations at any time throughout the story, though not at all time. Because of spatial and temporal constraints, there is a need for a real-time cinematic system that will present to the spectator the on-going situations that are most relevant to the story at the time. In traditional film production, the director is responsible for the overall decision-making process. He ensures that the narrative is conveyed effectively using the film techniques at his disposal. In our system, the virtual director has a similar role. The virtual director will query the client narrative application and determine in real-time which idiom will best fit the scene, based on the events specifications. To do this, the director requires some information from the application, namely the type of event, number of participants, and emotional or affective context of the story at the current point in its telling. This paper refers to the specificities of real-time cinematic sequences, being the dynamic nature of stories instantiated from independent representations of character-based roles, and the difficulties in producing a meaningful montage of on-stage events generated in real-time, opposed to scripted film sequences. In the next sections, we will introduce the important concepts of character-centred storytelling as well as a brief description of our interactive storytelling system. Then, we describe the traditional cinematographic elements used in automatic camera control systems. Finally, we give an overview of the implementation of our camera system, illustrated by an example based on a typical situation of a conversation between characters. CHARACTER-CENTRED NARRATIVE REPRESENTATION We have chosen to use plans to represent individual roles for the characters, rather than the global narrative structure. Each character is associated a plan corresponding to its role or, more precisely, the set of possible role instantiations according to a given storyline. It can be seen as a resource for story generation. Each plan corresponds to the character’s role in a given story instantiation: it represents the plot through a character’s behaviour (Figure 1). The plot itself consists in the on-stage integration of the various roles through the situations created by the interactions between characters. The on-stage performances of characters are the translation of each sub-task element of the character’s plan, called primitive actions, and perceived from the spectator’s point of view as a sequence of meaningful events. These actions are formally represented as syntactic triples (subject, verb, object). The subject represents the protagonist (the actor), the verb corresponds to the action to carry out, and the object represents either a physical object or another character. Though the action can not exist without the subject (actor), it may be performed without the specification of an object. Figure 1: Plan Representation of Character Behaviour© SCS While there are no straightforward rules to convert high-level narrative functions into characters’ plans, we have attempted to devise specific rules that could be applicable in the context of the simple genre (sitcoms) with which we are experimenting. The basic hypothesis is that the final story will emerge from the relations that exist between the various characters’ plans, these relations being determined from the story genre. SPACE, TIME AND CAUSALITY Narrative is a way of comprehending space, time, and causality. Since in film there are at least two important frames of reference for understanding space, time, and causality, narrative in film is the principle by which data is converted from the frame of the screen into a diegesis - a world - that frames a particular story, or sequence of actions, in that world; equally, it is the principle by which data is converted from story onto screen (Branigan 1992). An important concept in interactive storytelling is causality as a story is defined as a sequence of causally related events. The editing must assure that the consistency of such causal chain of events remains (Raskin, 1998). Causality supports the consequences of interaction, whether it be character-character interaction or spectator intervention. Some interactive storytelling systems make causality explicit in their representations (Young, 2000). However, in a task network representation based on actions and sub-goals, causality is not explicitly represented. One form of implicit causality is the enabling of further actions by their predecessors in the task network ordering, but it is not related to interaction and dynamic generation. Other forms of causality are implicit, illustrated by the interplay of choice and causality in the narrative, which has been described by Raskin (1998). Many on-stage objects have an intrinsic narrative significance as being resources for characters’ actions. In modern narratology (Barthes, 1966), they refer to as a “dispatcher”: a dispatcher is an object to which choice is associated, triggering narrative consequences. In the previous section, we described characters’ actions as represented by triples (subject, verb, object). The potential influence of action resources on the unfolding story will emphasise the narrative implication of certain object