Evaluating the Authorial Leverage of Drama Management Sherol Chen,1 Mark J. Nelson,1,2 Anne Sullivan,1 Michael Mateas1 1 Expressive Intelligence Studio University of California, Santa Cruz {sherol, anne, michaelm}@soe.ucsc.edu 2 School of Interactive Computing Georgia Institute of Technology [email protected] Abstract A drama manager (DM) monitors an interactive experience, such as a computer game, and intervenes to shape the global experience so that it satisfies the author's expressive goals without decreasing a player's interactive agency. Most research work on drama management has proposed AI architectures and provided abstract evaluations of their effectiveness. A smaller body of work has evaluated the effect of drama management on player experience, but little attention has been paid to evaluating the authorial leverage provided by a drama management architecture: determining, for a given DM architecture, the additional non-linear story complexity a DM affords over traditional scripting methods. In this paper we propose three criteria for evaluating the authorial leverage of a DM: 1) the script-and-trigger complexity of the DM story policy, 2) the degree of policy change given changes to story elements, and 3) the average story branching factor for DM policies vs. script-and-trigger policies for stories of equivalent quality. We present preliminary work towards applying these metrics to declarative optimization-based drama management, using decision-tree learning to capture the equivalent trigger logic for a DM policy. Introduction Technology can expand the possibilities of narrative both for those who experience and those who tell stories, in particular by making narrative be interactive. Authoring interactive narratives, however, has proven quite challenging in practice. Narrative in games, although sharing some qualities with non-interactive storytelling, delivers a highly interactive experience, which requires new ways of approaching authoring. Traditional approaches to authoring interactive stories in games involve a scripted and heavily linear process, and extending this process to large stories with complicated interactivity is difficult. Drama managers provide an alternative approach, by allowing the author to assume a system that knows something at run-time about how to manage the story. Such approaches, however, are difficult to evaluate from the perspective of authors looking for reasons to use a drama manager rather than traditional authoring approaches. Authorial leverage is the power a tool gives an author to define a quality interactive experience in line with their goals, relative to the tool’s authorial complexity. It has been pointed out that the “burden of authoring high quality dramatic experiences should not be increased because of the use of a drama manager” (Roberts & Isbell, 2008), but determining whether that is the case depends on determining both the complexity of an authoring approach and the gains it provides. Previous work has studied how experience quality can be improved by drama management. This does not directly imply an authorial benefit, however. To show that, there needs to be some reason to believe that traditional authoring methods could not have achieved the same results, or that they would have required considerably more effort to do so. A way to get at that comparison is to look at the set of traditional trigger-logic rules that would be equivalent to what a drama manager is doing. We propose three criteria for evaluating the authorial leverage of drama managers in this manner: equivalent script-and-trigger complexity of their policies, policy change complexity, and average branching factor of their policies. We present preliminary work applying these metrics to declarative optimization-based drama management (DODM), by examining the equivalent trigger-logic for a drama-manager policy as captured by a decision-tree learner. Drama Management In this work, we focus on DODM, an approach to drama management based on plot points, DM actions, and an evaluation function (Weyhrauch, 1997). Plot points are important events that can occur in an experience. Different sequences of plot points define different player trajectories through games or story worlds. Examples of plot points include a player gaining story information or acquiring an important object. The plot points are annotated with ordering constraints that capture the physical limitations of the world, such as events in a locked room not being possible until the player gets the key. Plot points are also annotated with information such as where the plot point happens or its subplot. The evaluation function, given a total sequence of plot points that occurred in the world, returns a “goodness” evaluation for that sequence. This evaluation is a specific, author-specified function that captures story or experience goodness for a specific world. While an author can create custom story features, the DODM framework provides a set of additive features that are commonly useful in defining evaluation functions (e.g. Weyhrauch, 1997; Nelson & Mateas, 2005). DM actions are actions the DM can take to intervene in the unfolding experience. Actions can cause specific plot points to happen, provide hints that make it more likely a 20plot point will happen, deny a plot point so it cannot happen, or un-deny a previously denied plot point. When DODM is connected to a concrete game world, the world informs the DM when the player has caused a plot point to happen. The DM then decides whether to take any actions, and tells the world to carry out that action. Given this model, the DM’s job is to choose actions (or no action at all) after the occurrence every plot point so as to maximize the future goodness of the complete story. This optimization is performed using game-tree search in the space of plot points and DM actions, using expectimax to backup story evaluations from complete sequences. Measuring Authorial Leverage The evaluation of DODM thus far has established at least preliminary positive results for: the technical features of optimization (Weyhrauch, 1997; Nelson et al, 2006; Nelson & Mateas, 2008), the effect on player experience (Sullivan, Chen, & Mateas, 2008), and the correspondence of some evaluation functions to expert notions of experience quality (Weyhrauch, 1997). None of this establishes the usefulness of DODM for authors, however, if