Designing User Interfaces for Games
Zhan Ye Human-Computer Interaction Institute Carnegie Mellon University December, 2000
Introduction We have seen a lot of articles about artificial intelligence (AI) programming, path-finding algorithms, and 3D modeling in various books and magazines. Given the fact that the game market is so competitive, almost every aspect of game design and development has been studied carefully to find better ways to create better games. But it is sad to see that one important aspect of game design has been long neglected - user interface (UI). Nobody has ever mentioned the role and importance of UI to a game. Nor have we seen any research on how to design a better UI for games. It is interesting to notice that other software industries have all invested a lot of time and effort in finding new methods and processes to design and evaluate UI. Just look at those online job sites, you will find a lot of job openings for UI designers, interaction designers, usability specialists and user experience designers. But in the game industry, UI designers still don’t have an important role; in fact in most teams they don’t even exist. Even in big companies that have realized the problem, such as Microsoft, their attempt to integrate UI design into the whole game design and development process is still in its early stage and hasn’t shown any promising results yet. Game designers still ask fundamental questions like “Why does user interface matter?” In the first part of this article, I will try to answer this question. In the second part, I will briefly describe the differences between the UIs of PC and console games. In the third part, I will give some ideas about why and how to use human-computer interaction (HCI) methodologies to improve UI design for games and how to integrate these methods into the game design and development process. In the final part of the article, I will give a detailed example using The Age of Empires II from Microsoft to show how HCI methods can be used to help game designers locate design problems and create innovative designs.
Why User Interface Matters User Interface: What Gamers Actually See and Interact With There are thousands of games coming out every year, which can be classified into 5 to 7 different genres. But no matter what genre games belong to, User Interface whether they are action games or RPG games, strategy games or sports games, they all share the Game same structure. From a system architect’s point of Core view, a game is a three-layered system. Figure 1 shows this layered system. For purpose of this paper, we call the inner layer the Game Core, and it represents the mechanics of the game. It defines the rules of a game, for example, how high the character can jump, how many enemies he will encounter in this level and where they are located. The Game Figure1. Game as a layered system. Core is designed by game designers who have a perfect mental model of the game. Gamers, however, can not access this layer directly, and therefore they can never get a complete mental model of the internal mechanics of a game. If they do, playing the game would be no fun at all. To keep them away from the Game Core, game designers wrap it with another layer, which is the UI layer. The only way gamers can get knowledge of the Game Core is through the UI layer. It includes both hardware elements (mouse, keyboard, controller) and software elements (menus, buttons, icons.) The UI layer is a medium between the Game Core and the gamers. Gamers have complete access to the UI layer and they have to be knowledgeable about this layer to successfully play the game. The outermost layer is the Gamers layer. We include gamers in this model because good UI is not a function of the software, but the property of the user. Without gamers, the system is not complete and cannot be evaluated. Gamers
User Interface Defines Game Play Most game designers don’t realize the fact that user interface can define game play. It is understood that game play is the most important property of a game. No matter how beautiful the 3D images are, or how involving the story is, without good game play a game definitely cannot succeed. But game play is a rather vague concept and hard to describe. Software engineering principles tell us that we cannot improve a property of a software product unless we can define and measure it. So it is very important to find a way to define game play. One possible way is to define it is by the way gamers interact with games; that is, through the UI and the interactions associated with it. By doing that, we will able to analyze game play, not as an abstract concept, but something concrete that can be described, measured, and studied.
User Interface Makes Games More Challenging
For RPG games, like the Final Fantasy series, you don’t want gamers to see the whole world map when they start the game. Instead, you want them to explore the world piece by piece. The fun of playing games largely depends on the exploration and the excitement and uncertainty it brings. For the same reason, we don’t want gamers to know the internal mechanics of a game when they start playing the game. We want them to gradually discover how the game ‘tricks’ them and what kind of strategies they should use. Through trial and error, they will build an increasingly complicated mental model of the game and feel a sense of accomplishment because their knowledge and skills are improving over time. So UI doesn’t only help gamers to play games successfully, but also controls the pace at which the internal mechanisms will be revealed to gamers. Eventually the internal mechanics will be revealed, and gamers will be able to finish the game successfully.
User Interfaces of PC and Console Games Differences between the User Interfaces of PC and Console Games
Figure 2. Console/Arcade Games like the Street Fighter series use minimum GUI elements. Their UIs are designed around specific hardware – the controllers.
Figure 3. We can find a lot of conventional GUI elements (icons, buttons, sliders, status bars, tabs) in the UIs of PC games like the Civilization series. Some games, such as the DOS version of Transport Tycoon, even created their own windows systems.
There are many differences between the UIs of PC and console games. The most important difference, however, is that the games use different input/output (I/O) devices. PC games heavily rely on the combination of keyboard and mouse. Whereas console games have controllers, which are more refined, although limited, input devices. The form factors of the hardware, especially the I/O devices, greatly affect the design of UI. From a gamer’s point of view, the UIs of PC games are more obvious than those of console games. Most gamers realize that they are interacting with some kinds of user interfaces when they are playing PC games such as The Age of Empires and C&C. Just look at the familiar icons, menus, and windows. But gamers playing console games sometimes don’t realize that they are interacting with the UIs. Can you ask a gamer what kind of UI Super Mario 64 has? UIs of console games are hidden deeply in the careful calculation of the usage of controllers, which is not obvious from the screen. We can call the UIs of PC games software-oriented interfaces, which means they utilize lots of conventional GUI elements to represent actions. The UIs of console games, on the other hand, are hardware-oriented interfaces, which means they are designed around the form factors of controllers and use relatively fewer GUI elements.
Why Transported Games Rarely Succeed The reason why most transported games fail is very simple. The original game succeeds because of its game play, which largely relies on the UI design. In order to transport a game to another platform, game designers are facing totally new I/O devices. They have to redesign the UI since it is the form factors of the I/O devices that determine what kind of UI will work. By doing this, the original, successful, proven game play will be affected. There is no guarantee that the new game play will be as good as the original one since they are inherently different. Transporting a game to a different platform
should be considered one of the major challenges that the UI designers face. The amount of work required would equal, or even surpass, designing a UI for a new game.
Figure 4. To solve the transporting problem, The Playstation version of Transport Tycoon even included a mouse with the software. But even that didn’t help much and the Playstation version still didn’t do as well as the PC version.
Why Use HCI Methods to Guide UI Design Cannot Rely Solely on Our Intuition In the current game industry, most game designs are guided only by designers’ intuition. Game designers are seasoned gamers; they have a feeling for what works in games and base their designs on this. There may be a few geniuses in the world who can do things right by just using intuition, but for most of us, we cannot rely on our intuition alone. The quality of games can certainly not be guaranteed this way. We need formal methods and processes to guide, evaluate, and help us improve our designs. Sooner or later, game designers will realize the limitations of their intuition and the ad hoc solutions they are using now, and admit the necessity of incorporating formal methods and processes into game design and development.
HCI Is the Closet Research Field to Game Design Game designers need formal methods and processes in order to create high-quality games. But where can they get them? Game designers don’t have enough time to research and design methods and processes from scratch. What they can do, however, is to look at related research fields and other software industries and borrow ideas from them. HCI is the closest research field to game design. It is the study of how people use computers, and how to design, implement and evaluate computer systems so that can be used easily, effectively, and enjoyably. UI is one of the major research areas of HCI. Researchers have developed a number of methods and processes over the past 10 years, some of which have been successfully used in other software industries and proven to be very useful tools for UI designers. Among these methods, heuristic evaluation (HE), cognitive walkthrough (CW), and Fitt’s law are especially valuable and can be adapted for use by game UI designers.
User Experience and Game Play I am curious about why UI has been long ignored by most game designers. It is interesting to compare the game industry with other software industries. It wasn’t until several years ago that some software companies began to realize the importance of UI. Since then, user experience, as a newly invented "magic word", has been talked a lot. User experience is the high level emotional impact on the users by the visual elements, information architectures, navigation, and interactions of a UI. Delivering an enjoyable experience to users while they are using software is the ultimate goal of HCI research and applications. Now many software companies have established their user experience groups and started using formal methods and processes to achieve this goal. If we look at the game industry, we will be surprised to see that it is way ahead of other software industries in realizing the importance of the general user experience. The game industry even invented a unique name for user experience for games, which is game play, from its very first day since the early '80s. The game play in a game is comparable to the user experience to a Web site or business software. The game industry coined the magic word
almost 10 years before other software industries. But the game industry just stopped there, without any conscious attempt to find out formal methods and processes to help game designers to create good game play. Game designers still highly rely on their intuition to do their work, whereas Web designers have a number of well-developed methods and tools to use.
How to Apply HCI Methods to UI Design Get Average Gamers Involved Most UI designers know the HCI mantra, “You are not like the user.” What it says is that nobody, especially domain experts or usability experts, can replace the role of users in user tests. Designers cannot evaluate their own designs effectively since they are not actual end users. For example, in the business software industry, developers usually don’t use the software they develop. Before they start a project, they have no indication towards what the software will be used for. It makes sense to hire real users to do user research and user tests. But in the game industry, the situation is quite different. Most game designers are avid gamers themselves; they know a lot about games. Therefore, it seems as if there is no need to get average gamers involved in the game design process. But this is wrong! We still need average gamers. There are two reasons for this: first, game designers are too skillful. Their knowledge about games is way beyond that of average gamers. Ironically, this will prevent them from finding obvious design problems. Things that they think are too easy may be too tricky for average gamers. We must bear this in mind – expert gamers only represent a small fraction of the whole gamer population and we don’t want to design games only for them. Second, game designers know too much about the design. Their mental model of the game is almost perfect because they designed it. Average gamers can never achieve this. They have their own simple, incomplete, sometimes incorrect, mental model. The mental model determines the user behavior. This means that the user behavior of game designers will be different from that of average gamers. Due to these two reasons, game designers, though they are very knowledgeable about games, cannot replace average gamers in the game design process. We need average gamers to evaluate and test our designs. The results we get from these evaluations and tests will prove to be unbiased and invaluable.
Adapting HCI Methods for Game UI Design We’ve mentioned that there are a number of HCI methods that have been proven to be useful and can be adapted for game UI design. The following three are particularly suitable for game UI design. •
Cognitive Walkthrough (CW): CW is an inspection method that focuses on evaluating a UI design for ease of learning by exploration. Its focus is motivated by the observation that many users prefer to learn software by exploration. This is often the case for games. When gamers start playing a game, they generally know nothing about it. They learn how to play and use the interface by trial and error. When game designers design a game, they should be aware of this, and should design a game that can be successfully played by exploration. CW can be used to quickly go through the game design and determine whether or not gamers can do it. Furthermore, this method doesn’t require a matured prototype since it is a mental process.
•
Think-Aloud Protocol: This method can be used to observe gamers playing an existing game. Essentially, it requires that UI designers ask gamers to “think aloud” as they play games. It can help us to know what’s in gamers’ minds, the strategies they use, their expectations, frustrations, and confusions. Some people consider think-aloud protocol “may be the single most valuable usability engineering method” because designers now can discover things inside gamers’ mind.
•
Fitt’s Law: In addition to these two methods, another powerful tool that can be helpful is the Fitt’s law. It is a very important theory that, once applied, can greatly improve the efficiency of a UI design. It states that the time required to move the cursor to a target size S which lies a distance D away is given by T=Im Log 2 (D/S + 0.5). The longer the distance, the smaller the target size, and thus more time will be needed to click the target, as the consequence, the user will feel more stress. The general design goal, then, is to reduce the distance between two controls and enlarge the size of the controls under given constraints. Designer should also avoid device swapping and disturbing the focus of attention. Fitt’s Law is extremely valuable not just because we can now evaluate two alternative designs quantitatively, but also because it provides a guideline and perspective for us when we think about design.
Problems of Rapid Prototyping and Iterative Design A widely used design approach in other software industries is the combination use of rapid prototyping and iterative design. Designers can use HTML, Director, or Visual Basic to build a prototype, only implementing the front-end interface without worrying about the back-end functionality, then they test the prototype with some users. After getting useful feedback, they make adjustments to the initial design and enter another round of iteration. This approach has been proven to be very successful in other software industries. But it may not work that well for games. Games need precise controls and timing, whereas business software does not. You don’t need to worry about response time when you build a prototype for business software. But you cannot ignore it when building a prototype for a game. A simple prototype cannot faithfully represent the real game play that the final game will have. A complicated prototype may be too expensive to build. Comparing to business software, it is also much harder to separate UI with back-end functionality of a game. So far there is no easy solution for this problem. Perhaps game designers need some kinds of tools like Director, but with more interaction features, that they can use to build prototypes.
UI Design Guidelines User Interface Shouldn’t Be Distracting One common mistake that we have seen in a lot of games, even successful ones, is that the UI is so beautiful, so overly decorated that it becomes a distraction. UI should have aesthetic taste, but more importantly, it should be simple, efficient, easy to use, and consistent with the whole game environment.
Figure 5. The UI of Iron Helix is a hyper-realistic and highly literal recreation of a real-world console. Notice the details such as the damage and corrosion. But the question is, “Do gamers really want to look at the UI, or they want to immerse into the beautiful outer space?” The UI occupies almost ¾ screen space and leaves little room to display the virtual world the gamers really i nterested in. This is one extreme example of how UI can be distracting and overly weighted.
User Interface Should Provide Enough Visual Affordance Many studies show that the majority of end users are very reluctant to read user manuals. They prefer to learn how to use software through exploration. This is also true for gamers. Never expect gamers read manuals, even for complicated strategy games like AOE! Design a UI with good affordance instead! By affordance we mean form affords certain actions and makes that apparent. A good UI should use its visual appearance to suggest its functions. Gamers should be able to understand and act easily by just looking at the UI.
Figure 6. AOE’s building buttons use the actual images of the buildings. They don’t provide enough affordance. They are visually complicated and look similar to each other (especially the first three in the first row and the last three in the second row.) They fail to provide a cognitive bridge between the buildings themselves (the i mages of the buildings) and the functionality of the buildings (what kind of products they produce.) Gamers, even experienced ones, cannot easily figure out which building is for what purpose and will hesitate when they need to choose among them.
Figure 7. In AOE II, the building buttons are greatly improved. They are elegantly simplified icons. Although simplified, they represent their functions clearly. Because instead of using the actual images of the buildings, UI designers focused on the functionality of the buildings, that is, the products the buildings produce. Gamers will have no problem choosing the one they want among them. The new design is more efficient since it can lower the cognitive load of the gamers when they play games.
User Interface Should Be Balanced The visual elements (buttons, labels) in a UI should be properly arranged, sized, and aligned to reinforce logical relationships among them and ensure the UI is stable on the screen. Most game UI designers pay little attention to these details, and they just put controls and labels everywhere possible.
Figure 8. The display panel in AOE II is unbalanced, although the labels and icons are aligned. It is overly weighted in the left side and leaves plenty of e mpty space in the lower right-hand corner.
User Interface Should Be Transparent A good UI should be transparent. Gamers should be able to forget about the menus, buttons, icons, and windows and immerse into the virtual world that the game creates. In that sense, UI designers’ job is to design UIs that are transparent – UIs that can be used so naturally by the gamers so that nobody would even notice its existence. I strongly believe that it is the ultimate goal of UI design.
Case Study: Age of Empires (AOE) II is a strategy game published by Microsoft. It is well received and praised by both gamers and game designers because of its innovative graphics and involving game play. But even a successful game like AOE II still has its problems, and using it as an example will enable us to realize that there are still many areas that can be improved. In order to evaluate AOE II’s UI, the first step, we need to find some tasks that are frequently performed by gamers. Each task is an action sequence that is performed step by step by the gamer to achieve a specific goal.
The Action Sequence of Constructing a Building One of the most frequently used tasks in AOE II is constructing buildings. This action sequence includes 5 major steps, selecting a villager, moving to the building panel, selecting a building type, moving to the intended location, and clicking to “drop” the building.
Figure 9. The action sequence of constructing a building. Red lines with arrows are used to show the path of the cursor movement. Before we analyze the action sequence, we need to make some assumptions about the initial positions of the villager and the cursor. Let’s assume that we have a villager standing in the center of the screen, and the initial cursor position is also at the center of the screen. Step 1: The gamer clicks the villager. The building panel will show a number of available actions.
Step 2: The gamer moves the cursor down to the panel.
Step 3: The gamer chooses one building
Step 4: The gamer moves the cursor to the edge of the screen to scroll the map.
Step 5: Then the gamer moves the cursor back to the center of the screen to “drop” the building.
Action sequence is done.
Pop-up Menu Can Improve the Efficiency of the UI The above action sequence, although it looks pretty complicated, is just a sequence of mouse movements and clicks. You move the mouse to place A, click, then move it to place B, then click, and so on. The time the gamer spend on moving the mouse is a lot longer than that of a simple click. According to Fitt’s Law, the longer the moving distance is, the longer the action sequence takes, the more muscle fatigue gamers will feel. Cursor movement itself doesn’t produce any direct results. The most obvious way to reduce the moving distance is to use a pop-up menu. As Figure 10 shows, when the gamer clicks a villager, a menu will pop up near the villager, the gamer can easily choose one building from the pop-up menu. Comparing the distances between the two designs, the new one may save less than 0.1 second every time. Sounds pretty trivial; but think
(a) (b) Figure 10. Comparing to the old design (a), the new design (b) greatly reduces the required mouse movement.
about it: gamers usually have to do the same action sequence hundreds of times when they play the game. The new design saves a lot of time, and much more importantly, relieves the muscle fatigue greatly. It is not only an efficient design, but also helps to relieve the pain and stress associated with computer usage to some extent. The new design (b) also saves a lot of screen space. The building panel occupies a huge amount of valuable screen space, and it is empty most of the time. Changing it into a pop-up menu leaves more screen space for the world map. Research shows that the sense of immersion is very important to games and VR systems. Having a large amount of screen space devoted to UI will reduce the sense of immersion, since UI is not an organic part of the virtual world of the game. UIs should occupy the minimum amount of space required to display the most important controls and information.
Two-Handed Interaction as a Hardware Solution to Map Scrolling The new design we proposed early only solves the problem of the long mouse movements required to construct a building. From the analysis of the action sequence, we noticed that map scrolling also requires a lot of mouse movements. A gamer has to move his mouse to one of the edges of the screen and stay there to scroll the map. Does this remind us of Web browsing? You move your mouse to the right edge of the screen
to click on the scrollbar to scroll the Web page. They all require long mouse movement to the edge of the screen. To solve this problem, Microsoft provided an innovative design, IntelliMouse, which is a mouse with a scroll wheel. It is a hardware solution that totally eliminates the required movement. The D in Fitt’s law is reduced to zero, which means the efficiency is greatly improved. Similarly, we can use a hardware solution for the map scrolling in AOE II. In the real world, people perform tasks with both hands, because it is more efficient. We often use our non-dominant hand to hold objects while using our dominant hand to perform precise operations on them. Most of today’s UIs still make use of only one pointing device that users manipulate with their Figure 11. Microsoft IntelliMouse is an dominant hand. A lot of research, however, excellent hardware solution to the device has shown that carefully designed two-handed swapping/mouse movement problem. interaction can improve the efficiency of UIs. The most famous example is Xerox PARC’s Toolglass. I will not describe the details here, but basically they used two pointing devices, one mouse for the dominant hand, one trackball for the non-dominant hand. The nondominant hand controls the position of the Toolglass using trackball because it requires relatively lower precision. The dominant hand controls the cursor. We can adapt this design Figure 12. Trackball is ideal for map for AOE II using a trackball to control the scrolling because of its form factor. scrolling of the map and a mouse to control the cursor. The benefits of this design are, first, the required cursor movement is zero. It is basically a hardware solution to the problem. Second, by using two pointing devices, we introduce parallel interaction into the game. Now gamers can move the cursor and map concurrently. The steps 2 to 5 in the action sequence will be changed from being serialized to parallel and it saves a lot of time.
Conclusion The above case study explains how to analyze a UI and game play by studying its action sequences. It also shows how to use design guidelines and Fitt’s law to find problems in the action sequences. The proposed two-handed interaction shows that HCI research can bring innovative ideas to game designs. Although we need time and real-world testing to determine if the ideas innovated in the labs can succeed and eventually benefit us, it is always important to have these innovative ideas first.
