Presentation Abstract

Graphical user interfaces (GUIs) have become nearly ubiquitous as a means of interacting with software systems. The widespread use of GUIs is leading to the construction of more and more complex GUIs. With the growing complexity comes challenges in testing the correctness of GUIs and the underlying software. Some of the important challenges include test¡case generation, test¡ oracle creation, and regression testing.

Current GUI testing practices involve a significant amount of manual effort on the part of the test designer. Most test designers employ tools that automate certain aspects of the testing cycle. Most common among such tools are capture/playback tools used to capture the user events and GUI screens during an interactive session. The recorded sessions are later played back whenever it is necessary to recreate the same GUI states. These tools generally store information at a low level of abstraction, capturing actual mouse positions, button clicks and storing bit¡maps. Representing the information at such a low level of abstraction makes it difficult to tailor the recorded session for other test cases. A popular alternative to using capture/playback tools is to program a test case generator. Programming requires that the test designer program all possible decision points present in the GUI. However, this approach is time consuming, and is susceptible to missing important GUI decisions. Moreover, the expected output must also be determined by the programmer.

Planning Assisted Tester for grapHical user interface Systems (PATHS) is a research project designed with the primary goal of facilitating the automation of GUI testing. PATHS is based on planning -- a well developed and used technique in Artificial Intelligence (AI). Given a set of operators, an initial state and a goal state, a planning system produces a sequence of operators that will transform the initial state to the goal state. The key idea of using planning as the core of PATHS is that the GUI test designers will often find it easier to specify typical goals that users of the GUI software might have than to specify sequences of GUI actions that users might perform to achieve those goals. Thus we cast GUI testing as an instance of planning. PATHS has the goals of generating test cases automatically, incorporating oracle information into the testing process, and then automatically generating a regression test suite when re¡testing is done.

Currently, the PATHS system is capable of using GUI specifications to automatically generate a test suite, exploit the planning model to create test oracles, execute the test cases and pinpoint errors in the GUI. A test designer provides PATHS with a description of the GUI (e.g., in the form of GUI speci¡ fications). PATHS analyzes this description and creates an abstract model of the GUI and returns a list of operators from the model to the test designer. By using knowledge of the GUI, the test designer then defines the preconditions and effects of the operators in a simple language provided by the planning system. The test designer then triggers test¡case generation and oracle creation by describing scenarios (tasks), represented by a set of initial and goal states. PATHS generates a test suite for the scenarios. The test suite also has the oracle information built into it. During testing, the GUI exerciser uses this oracle information to compose the expected GUI state. The GUI exerciser compares the expected GUI state with the actual state to determine if the GUI's run¡time behavior is correct. After testing, identified problems in the GUI are corrected. Modifi¡ cations then lead to regression testing. The test designer describes the changes in the GUI, and PATHS generates the regression test suite.

The key to efficient test¡case generation, oracle creation and maintenance is PATHS's hierar¡ chical decomposition of the GUI. PATHS models a GUI hierarchically with high¡level operators that decompose into sequences of lower level ones. More specifically, PATHS begins the modeling process by partitioning the GUI events into several classes: (1) Menu¡open events that open menus, i.e., they expand the set of GUI events available to the user, (2) Modal events that open modal windows (i.e., no other window in the program can be interacted with until this dialog is closed), (3) Non¡modal events that open non¡modal windows, and (4) System¡interaction events that interact with the underlying software to perform some action. These event classes are then used by PATHS to create two types of planning operators -- system¡interaction operators and abstract operators. The system¡interaction operators are derived from the system¡interaction events. Abstract operators are more complex and are responsible for driving the hierarchical planning algorithms used by PATHS. The basic idea behind an abstract operator is that it represents a sequence of GUI events that invoke a modal window. The abstract operator encapsulates the events performed on the modal window by treating the interaction within that window as a separate planning problem. Abstract operators are decomposed into lower level operators by an explicit call to the planner. PATHS uses the system¡interaction and abstract operators to generate the test suite.

PATHS automatically generates test oracles to check the GUI's state during and after test execution. The key to oracle creation is to have oracle information integrated into the test case so that the GUI exerciser knows what to check at each step. One of the primary motivations of using planning for GUI testing is that much of the state can be recovered directly from the planning model. During plan construction, the planner keeps track of the intermediate states of the GUI. PATHS extracts these intermediate states and integrates them with the generated test case to act as oracles.

We are currently extending PATHS with efficient regression testing techniques that draw on planning. The hierarchical decomposition of the GUI done by PATHS allows identification and isolation of GUI components that were modified. Changes are then made only to the effected test cases and oracle information. Moreover, hierarchical planning aids in retaining most of the test cases and oracle information defined at the higher levels of abstraction. Changes are made only at the lower levels.

About the Author

Atif M. Memon is a Mellon Fellow at the Department of Computer Science, University of Pittsburgh. He received his BS and MS in Computer Science in 1991 and 1995 respectively. He was awarded a Gold Medal for First Position in BS. He enrolled at the University of Pittsburgh in 1996 and is currently a Ph.D. candidate. In 1999, he was awarded a Fellowship from the Andrew Mellon Foundation for his Ph.D. research. His research interests include program testing, software engineering, artificial intelligence, plan generation, and code improving compilation techniques. He is a member of the ACM and the IEEE Computer Society.

More information can be found at the Author's Home-page