Extracting Usability Information from User Interface Events.

1. INTRODUCTION

User interface events (UI events) are generated as natural products of the normal operation of window-based user interface systems such as those provided by the Macintosh Operating System [Lewis and Stone 1999], Microsoft Windows [Petzold 1998], the X Window System [Nye and O'Reilly 1992], and the Java Abstract Window Toolkit [Zukowski and Loukides 1997]. Such events indicate user behavior with respect to the components that make up an application's user interface (e.g., mouse movements with respect to application windows, keyboard presses with respect to application input fields, mouse clicks with respect to application buttons, menus, and lists). Because such events can be automatically captured and because they indicate user behavior with respect to an application's user interface, they have long been regarded as a potentially fruitful source of information regarding application usage and usability. However, because user interface events are typically extremely voluminous and rich in detail, automated support is generally required to extract information at a level of abstraction that is useful to investigators interested in analyzing application usage or evaluating usability.

While a number of potentially related techniques have been applied to the problem of analyzing sequential data in other domains, this paper primarily focuses on techniques that have been applied within the domain of HCI. Providing an in-depth treatment of all potentially related techniques would necessarily limit the amount of attention paid to characterizing the approaches that have in fact been brought to bear on the specific problems associated with analyzing HCI events. However, this survey attempts to characterize UI events and analysis techniques in such a way as to make comparison between techniques used in HCI and those used in other domains straightforward.

1.1 Goals and Method

The fundamental goal of this survey is to construct a framework to help HCI practitioners and researchers categorize, compare, and evaluate the relative strengths and limitations of approaches that have been, or might fruitfully be, applied to this problem. Because exhaustive coverage of all existing and potential approaches is impossible, we attempt to identify key characteristics of existing approaches that divide them into more or less natural categories. This allows classes of systems, not just instances, to be compared. The hope is that an illuminating comparison can be conducted at the class level and that classification of new instances into existing classes will prove to be unproblematic.

In preparing this survey, we searched the literature in both academic and professional computing forums for papers describing computer-aided techniques for extracting usability-related information from user interface events. We selected and analyzed an initial set of papers to identify key characteristics that distinguish the approaches applied by various investigators.

We then constructed a two-dimensional matrix with instances of existing approaches listed along one axis and characteristics listed along the other. This led to an initial classification of approaches based on clusters of related attributes. We then iteratively refined the comparison attributes and classification scheme based on further exploration of the literature. The resulting matrix indicates areas in which further research is needed and suggests synergistic combinations of currently isolated capabilities.