Overview

While massive amounts of information become available in digital forms daily, human’s capabilities of comprehending, analyzing, and synthesizing information remain a constant. To help human users to cope with increasingly more information efficiently and effectively, we are building a wide range of tools that marries machine power with human intelligence. In particular, our work focuses on assisting users in access, viewing, navigate, and analyze large volumes of complex information in the context of a continuous, evolving information-based investigation process.

Unlike conventional WIMP-based interaction paradigms, our work supports an intelligent user interaction paradigm that helps users to express their complex information requests easily in multiple forms (e.g., natural language and visual query), while dynamically guiding the users in their tasks by providing them with various information interaction and analytic appliances (e.g., recommending a suitable data analytic method and visual appliance for examining the interested information). In addition, the system is able to dynamically track and constantly learn from user interaction behavior and then adapt itself to better helping users in their tasks. For example, if a user knows exactly what she is looking for, she could use natural language expressions to articulate her intent in context. If she does not happen to know the system’s vocabulary, the user may use a traditional GUI-based form to construct a request. During this process, the system will teach the user on the fly what suitable vocabulary she could use to express the similar requests in natural language. Such human-computer interaction sequences are also automatically recorded, from which interaction patterns are extracted and used for the system to learn the user’s vocabulary and improve its own interpretation capabilities.

Overall, the goals of our work are two-fold. First, we are developing intelligent user interaction technologies that can radically simplify user information tasks. Second, such technologies aim at supporting a mixed-initiative information access, analytics, and insight discovery process. To apply our technologies to a wide range of real-world applications, our work is also governed by two important principles: reusability and extensibility. The reusability principle ensures that
our work be reusable or easily customizable across different applications and domains. The extensibility principle governs that our technologies can be easily extensible to cover new functionalities or new domains. In general, our work falls in the following areas:

Semantics-based, Multimodal Request Composition and Interpretation

This area of work focuses on helping users to author their information requests, including complex analytic requests efficiently by leveraging multiple interaction modalities, such as natural language, GUI, and direct manipulation (e.g., interactive visual inquiry composition).

Smart Visual Analytics

This area of work focuses on helping users to employ visualization tools in their analytic tasks. In particular, it focuses on how to dynamically transform the raw data for better visualization, create context-sensitive visual summarization, and to recommend the suitable visual analytic tools in context. To better help users in their tasks, we are also developing technologies on how to keep track of and recognize user intent through their visual interaction behavior. Based on understood user interaction behavior, the system can then better recommend suitable visualization tools.

Automated and Dynamic User Context Modeling

This area of work focuses on representations and technologies on how to automatically maintain and manage users’ interaction behavior (e.g., discovery paths). The result of this work is also the core context being used for the system to learn user behavior and adapt itself to better assisting users when needed. For example, the system may dynamic recommend suitable analytic appliance or discovery paths in context to help users to accomplish their information discovery goals.

Intelligent Interaction Assistant

This broad body of work is concerned with how to best help users in their tasks during their interaction. With the vast amounts of information to wade through and the complexity of today’s information systems, users may not know what information they would like to access and analyze, and how to access and analyze such information. We are developing a wide range of smart technologies that can dynamically help users in such situations, ranging from automatically recommending inquiries to access data to exact information content pertinent to the users’ task at hand.