Toward a Deeper Understanding of the Role of Interaction in Information VisualizationJi Soo Yi, Youn ah Kang, John T. Stasko, Member, IEEE, and Julie A. JackoAbstract—Even though interaction is an important part of information visualization (Infovis), it has garnered a relatively low level of attention from the Infovis community. A few frameworks and taxonomies of Infovis interaction techniques exist, but they typically focus on low-level operations and do not address the variety of benefits interaction provides. After conducting an extensive review of Infovis systems and their interactive capabilities, we propose seven general categories of interaction techniques widely used in Infovis: 1) Select, 2) Explore, 3) Reconfigure, 4) Encode, 5) Abstract/Elaborate, 6) Filter, and 7) Connect. These categories are organized around a user’s intent while interacting with a system rather than the low-level interaction techniques provided by a system. The categories can act as a framework to help discuss and evaluate interaction techniques and hopefully lay an initial foundation toward a deeper understanding and a science of interaction. Index Terms—Information visualization, interaction, interaction techniques, taxonomy, visual analytics 1 INTRODUCTIONInformation visualization (Infovis) systems, at their core, appear to have two main components: representation and interaction. The representation component, whose roots lie in the field of computer graphics, concerns the mapping from data to representation and how that representation is rendered on the display. The interaction component involves the dialog between the user and the system as the user explores the data set to uncover insights. The interaction component’s roots lie in the area of human-computer interaction (HCI). Although discussed as two separate components, representation and interaction clearly are not mutually exclusive. For instance, interaction with a system may activate a change in representation. Nonetheless, the two components seem to compose the two fundamental aspects of Infovis systems, and it seems reasonable to consider what each contributes to an end-user’s experience.We argue that the representation component has received the vast majority of attention in Infovis research. A cursory scan of a recent conference proceedings or journal issues in the area will uncover many articles about new representations of data sets, but interaction is often relegated to a secondary role in these articles. Interaction rarely is the main focus of research efforts in the field, essentially making it the “little brother” of Infovis. In other words, it is overshadowed by the more noteworthy representation aspects. A few papers have mainly focused on the interactive aspects of Infovis (e.g., [10, 15, 25, 47]), but these are relatively uncommon when compared to papers introducing new data representations. Interaction is an essential part of Infovis, however. Without interaction, an Infovis technique or system becomes a static image or autonomously animated images (e.g., InfoCanvas [28]). While static images clearly have analytic and expressive value (e.g., [8, 29, 46]), their usefulness becomes more limited as the data set that they represent grows larger with more variables. Actually, even with a static image such as a poster, a user (or a reader) will often perform several interactions (e.g., rotating the poster, looking closer/further, and jotting down notes on the poster). Spence even suggests the notion of “passive interaction” through which the user’s mental model on the data set is changed or enhanced [38]. Finally, through interaction, some limits of a representation can be overcome, and the cognition of a user can be further amplified (e.g., [15, 29]). The importance of interaction and the need for its further study seem undisputed. For example, the recent book Illuminating the Path: The Research and Development Agenda for Visual Analyticscalls for further research on interaction: “Recommendation 3.3: Create a new science of interaction to support visual analytics. The grand challenge of interaction is to develop a taxonomy to describe the design space of interaction techniques that supports the science of analytic reasoning. We must characterize this design space and identify under-explored areas that are relevant to visual analytics. Then, R&D should be focused on expanding the repertoire of interaction techniques that can fill those gaps in the design space.” ([45], p. 76) This recommendation concerns visual analytics which is not equivalent to Infovis, but the two clearly share much in common and the motivation for this call can equally be applied to Infovis. While we believe that few would argue with the merits of the goals in the recommendation, precisely defining what is being called for is not so easy. What does it mean to create a “science of interaction” in visual analytics and Infovis? The recommendation speaks of developing a taxonomy of interaction techniques and identifying under-explored areas for future research. These are noble efforts, but we believe that a science of interaction also should involve gaining a deeper understanding of the utility and value of interaction in these fields. What does interaction contribute to the analytic process? For that matter, we might raise questions about the nature of interaction itself. In the context of Infovis, what is interaction and interactive behavior? Operations such as moving a dynamic query slider [3] to narrow the set of data points being shown or selecting an alternate point in a fisheye view [19] to change the focus seem like clear examples of interactive behavior. But consider a system where the user selects a menu operation to change from a scatter plot to a parallel coordinates of the data. Is that interaction? The purpose of this article relates to the recommendation from Illuminating the Path that was discussed above. Defining a science of interaction is a lofty goal and we do not purport to do so here, but we do seek to take some initial steps toward that goal. Our objective is to further current understandings of the role that interaction plays x Ji Soo Yi is with Health Systems Institute & H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, E-Mail: [email protected]. x Youn ah Kang and John T. Stasko are with School of Interactive Computing & GVU Center, Georgia