Abstract: In this note, I call for a move towards viewing interfaces as means for people and computers to collaborate on solving problems rather than means for people to control computers. This collaborative perspective on user interfaces can apply quite broadly, and not only provides a source for novel interface techniques but serves as a good tool for analyzing existing interfaces. The view affects thinking on interfaces primarily by motivating a different split in the roles and responsibilities of the two participants in problem-solving, the computer and the user.
In this note, I call for a move towards viewing interfaces as means for people and computers to collaborate on solving problems rather than means for people to control computers. This collaborative perspective on user interfaces [1] can apply quite broadly, and not only provides a source for novel interface techniques but serves as a good tool for analyzing existing interfaces. The view affects thinking on interfaces primarily by motivating a different split in the roles and responsibilities of the two participants in problem-solving, the computer and the user.
Many (though not all) computer-based activities can be thought of in terms of optimally solving some problem of interest to the user -- writing a (maximally) convincing memo, determining the (ideal) price for a product, constructing a (maximally) communicative diagram. Viewed as optimization tasks, such problems are typically hard, either in the informal sense of the word (it's hard to be convincing, for instance) or in the technical sense (the optimal price may be specified by a nonlinear program) or both. As a concrete example, consider the problem of constructing a diagram that communicates well some information in the form of a network diagram of nodes and links. This problem is hard in both senses, in that the aesthetic criteria are subtle, and combinatorial idealizations of the problem, such as graph layout with minimum edge crossings, have been shown to be NP-hard.
Computers are unlikely to be of much use in solving an informally or combinatorially difficult problem, so the standard paradigm is for the user to solve the problem and the computer to record the solution. One might use a word-processor to record the memo, a spreadsheet to record the computation of the price, or a drawing tool to record the diagram. But such an approach gives up far too easily; though the computer is unlikely to be of aid in solving the entire problem, the user's efforts can be leveraged by splitting the duties between the computer and user more equally leading to a more collaborative style of interaction. In particular, computers can be quite good at local optimization, whereas the user's mental effort is best expended on the global aspects of the problem.
Under this view, a key problem in user interface design is in defining a representation of the problem to be solved such that it can be naturally decomposed into a local portion that a computer would be good at and a global portion left to the user. In the case of the network-diagram layout example, computers are quite good at fine-grained placement of graphical objects whereas a user may find pixel-accurate placement, requiring fine-grained mouse gestures, extremely tedious. The user's role is more appropriately the coarse-grained layout, which is in fact the combinatorially more difficult part. Thus, an interface device that turns fine-grained into coarse-grained gestures, such as a snap-to-grid feature, exemplifies the collaborative interface view expounded here. Of course, using more information about the task can allow for even more of the burden to be placed on the computer. For instance, in a network diagram layout task, nodes are typically placed in standard positions relative to other nodes -- horizontally, vertically, diagonally, or radially aligned. An interface might leverage this fact by providing a snap-to-point feature where the points are defined not by a fixed grid but by the particulars of the current layout. Such devices are now seen in some drawing and design packages.
Our own work on collaborative interfaces for network diagram design has attempted to take this perspective even further, allowing the user to specify that nodes should be placed in certain configurations typical of network diagrams. The computer is used to locally satisfy these constraints while the user simultaneously structures the global layout. Both participants are acting at the same time to further the goals specified by the user, with the user doing the deep, combinatorial part and the computer the fine-grained, pixel-level placement. Thus, by adding even more semantic information about the task, an interface allowing even more equal collaboration can be built.
In general, this approach to interface design can be realized by viewing the task as a combinatorial optimization problem, and handing the local optimization part to the computer while the user performs the global part. This typically provides a way of using to best advantage the abilities of computer and user. The key to designing an interface then becomes representing the problem in such a way that this nice division of roles is feasible.
Note that using the computer for performing local optimization leads to a much different spin on optimization technology. For most other applications, one wants the optimization method most likely to produce the global optimum. Here, by contrast, one wants a method that is as predictable in its performance as possible, even at the expense of global optimality. Since we have already split up the optimization responsibility between the two parties, it is imperative that the user be able to intuit well how his or her actions will impinge on the solution under construction. For this reason, we have concentrated on using optimization techniques based (loosely) on simulating physical systems; users can have strong intuitions about how the interface will then behave based on analogy with the physical world.
This view of the desirability of fine-grained collaboration already underlies many good user interface devices and designs: they make use of task-specific information to leverage the actions of users, performing fine-grained details while leaving the high-level aspects to the user specifiable through gross interface gestures. But making the view explicit means that we can look for more opportunities for such collaborative interfaces, and push the ideas further, leading to problem-solving software tools that better leverage human performance through simple yet sophisticated user interfaces.
Citation: Stuart M. Shieber. 1996. A call for collaborative interfaces. ACM Computing Surveys, volume 28A (electronic), December.