1Visual Query of Multi-Dimensional Temporal Data Jerry Alan Fails, Amy Karlson, Layla Shahamat Department of Computer Science, University of Maryland College Park, Maryland {fails, akk, laylas}@cs.umd.edu ABSTRACT In this work we present an integrated interface for visual query and result-set visualization for search and discovery of temporal patterns within multi-dimensional data sets. Many have proposed visual query languages for temporal data but fail to address the presentation of the results. Others have designed successful visualizations of temporal data, but do not include support for ad-hoc query of interesting patterns. We couple both stages of temporal query in a single tool for exploration and discovery of temporal patterns. We define temporal patterns as sequences of Events with inter-event TimeSpans. Pattern queries allow for Event and TimeSpan attributes to be constrained in different ways. We formalize the types of pattern queries our system supports according to the levels of restriction imposed upon both Events and TimeSpans. Author Keywords Temporal queries, information visualization. ACM Classification Keywords H.5.2 [Information Interfaces and Presentation]: User Interfaces; H.5.2 [Information Systems]: Information Search and Retrieval — query formulation INTRODUCTION In many application domains temporal data queries are fundamental to understanding and reasoning over an information space. For example, in the medical field, time plays a critical role in assessing individual treatments based on personal medical history, as well as broader treatment success rates based on aggregate analysis over multiple case histories. Time also plays an important role in tracking and analyzing financial events that guide both investment and business strategies. In fact, most application domains depend upon temporal knowledge to understand or mine historic events. Despite this pervasive presence of temporal data, traditional database management systems do not provide explicit support for capturing the various types of temporal information that can be associated with an event nor do they support temporal reasoning over these events. In recent years systems have emerged to fill this gap, providing storage architectures coupled with access languages to support the types of rich temporal queries end-users would like to ask to improve their understanding of the data (e.g., [7, 12]). Even though progress has been made, database query remains a complex task for a vast majority of end-users for two main reasons. The first reason is a practical one — traditional database management systems require the user to be familiar with the structure of the data within the system in order to extract the desired information. This structure is typically not made available to end users, and when it is (e.g. via an SQL schema), retrieving data requires a formal specification that is unfamiliar to most users. Additionally, the structure required to describe the data may not necessarily conform to a domain expert’s model of the information, so mapping data requests from the user’s mental model to the storage model is not trivial. For the above reasons, many approaches have been explored to provide end-users with better access to and control of stored data, without requiring users to learn a formal query language, and often abstracting away the details of the storage architecture. Nearly all but the earliest examples use direct manipulation strategies to allow users to interact with representative objects to construct queries [5]. Visual query interfaces typically map directly to the underlying query language, attempting to retain as much of the original power of the language as possible, while making it much easier for the average user to formulate queries. Temporal databases that support rich temporal queries thus demand new visual interface features to support the extended capabilities and query requirements. Although the addition of a time dimension presents a challenge to visual query designers in maintaining the elegance and intuitiveness of a query system, it increases the expressive power of the results. We approach this problem domain by focusing on visual query and exploration of temporal patterns within medical histories. We define a temporal pattern as a sequence of Events together with the TimeSpans that separate them. In contrast to some systems that support time-series data, we consider the dimensionality of Events. We formalize the types of patterns our system finds and displays in terms of2the constraints placed on Events and TimeSpans. Within this framework, a representative task supported by our system would be finding and displaying patients who had high blood sugar on two consecutive blood tests within ten days. A pattern such as this would be challenging for an average user to formulate using a query language, or to interpret as a text-based tabular result. Our visual system facilitates both these stages of exploration and discovery. Although our initial prototype and running examples are modeled after a medical domain, we expect these techniques to apply equally well to many domains such as historical records (e.g., education registration, business) and transaction-based data sets (e.g., web logs, finance). RELATED WORK To provide historical perspective, we discuss the evolution from traditional, non-temporal relational databases, to those that explicitly manage and reason over time. We present associated interfaces for simplifying query to such databases, as well as introduce applications that have built over temporal repositories. Visual Query Interfaces to Relational Databases SQL is the standard query language for accessing data stored in relational databases. Due to the complexity of formulating SQL queries, several approaches have been developed to make database query more accessible for typical users. Perhaps the earliest approach was Query By Example (QBE) which presented the structure of the database as skeleton tables [21]. For the most basic queries in QBE, users simply indicated the attributes they wished to be returned by placing a mark in the appropriate data column. Users could also use constants to specify desired attribute values, and could bind results in different columns by using variables. Although QBE abstracts away the specifics of SQL syntax, it still required users to think of data in terms of the structure of the underlying database and