Beyond Screen Design

Creating good interfaces requires designers to think really hard about how they want to visualize the essence of the interface. In essence the interface becomes a visual language which in turn is created from several concepts: the basic visual representation, how data is visualized (information visualization and visual variables) , choosing appropriate metaphors, and providing good interaction techniques such as direct manipulation and dynamic queries.


  • Representations
  • Metaphors and direct manipulation
  • Visual variables
  • Methods for creating and developing interface ideas

Topics covered

  • representations
  • information visualization
  • visual variables
  • metaphors
  • direct manipulation
  • dynamic queries
  • Optional: Methods for creating and developing interface ideas

Required Readings

  1. Chapter 3: The Power of Representations. Norman, D. (1993) . In Things that Make Us Smart, 43-76, Addison-Wesley.
  2. Chapter 15: Information Search and Visualization. Shneiderman, B. (1998) 509-549. In Designing the User Interface, Addison Wesley.

Additional Readings

  1. Things that make us smart. Donald Norman (1993), Addison Wesley.
    • Chapters 3 & 4 give an excellent background and engaging discussion on the value of representations.
  2. Readings in Information Visualization, Card, Mackinaly and Shneiderman, Morgan Kauffman
    • the definitive source book for information visualization techniques
  3. Readings in Human Computer Interaction: Towards the Year 2000. Morgan-Kaufmann.
    • Working with Interface Metaphors, p. 147-151, talks about the roles of metaphors in the interface, and how designers can come up with good ones.
    • Visual Information Seeking, p. 450-456, introduces the notion of tight coupling of dynamic queries and starfield displays.
    • Tools and Techniques for Creative Design, p. 128-141, describes how we can apply traditions in existing disciplines to creative interface design, how metaphors and user observation can influence creativity, and how ideas can be generated.
  4. Interface metaphors and user interface design, Carrol, Mack and Kellog (1988),in Helander (ed.) Handbook of Human Computer Interaction, North Holland.
    • another valuable description of the role of metaphors
  5. Tufte, E. (1983) Visual display of quantitative information. Graphics Press, PO Box 430, Cheshire, Connectictut, 06410.
    • If you want to visualize data, this is a must read book just full of examples. His other books are also good:
    • Tufte, E. (1990) Envisioning information. Graphics Press, PO Box 430, Cheshire, Connectictut, 06410.
    • Tufte, E. (1997) Visual explanations Graphics Press, PO Box 430, Cheshire, Connectictut, 06410.
  6. Direct manipulation: A step beyond programming languages. Shneiderman, B. IEEE Computer 1992, 16(8).
    • The original article that coined the term and the concept of direct manipulation


The visualization videos demonstrate excellent examples of information visualization through animation, fisheye views, visual changes in large overviews of the data, and dynamic queries. The metaphor videos illustrate different metaphors and approaches to representations. These videos change year by year because there are so many good ones to choose from!


  • Information Visualization using 3D Interactive Animation, by Xerox PARC (1991, SVGR 63)
  • Browsing Graphs Using a Fisheye View, by DEC and Brown University (1993, SVGR 88)
  • High Interaction Data Visualization (1993, SVGR 88)
  • Dynamic Queries, by Shneiderman (1992, SVGR 77)
  • Visual information seeking using FilmFinder (1994, SVGR 97)
  • Hierarchical visualization with TreeMaps, by U Maryland(1994, SVGR 97)
  • Exploring large tables with the Table Lens, by Xerox Parc (1995, ACM CHI Video)
  • DragMag Image Magnifier, by Ware (1995, ACM CHI Video)
  • Toolglass and magic lenses: the see through interface by Bier (1994, SVGR 97)
  • A Taxonomy of See-through tools: the Video, by Bier (1995, ACM CHI Video)
  • The movable filter as an interface tool: the Video, by Bier (1995, ACM CHI Video)
  • Visualizing large trees using the hyperbolic browser, by Lamping (1996, ACM CHI Video)


  • Alternate Reality Kit, by Smith (1987, SVGR 26)
  • TeamRooms, by Roseman and Greenberg (1996, ACM CHI Video)
  • The DeckScape Web Browser, by Brown (1996, ACM CHI Video)
  • The Web book and Web Forager, by Xerox Parc (1996, ACM CHI Video)

In-class teaching tips

The presentation is fairly straight-forward. However, it is difficult to show a "recipe" for crafting good representations without seeing specific examples. To this purpose, I choose from the above videos to illustrate novel and creative interface designs and metaphors. Students tend to be quite impressed by the ideas presented in the videos.

Major sources used to prepare lecture material

  1. The collection Readings in Information Visualization by Card, Mackinaly and Shneiderman (Morgan Kauffman Press) is the definitive source book of relevant papers.
  2. The discussion about metaphors is taken both from the reading Working with Interface Metaphors, and from Interface metaphors and user interface design, by Carrol, Mack and Kellog 1988, in Helander (ed.) Handbook of Human Computer Interaction, North Holland.
  3. Chapters 3 & 4 of Donald Norman's book Things that make us smart provide an excellent background to the cognitive role of representations.
  4. Edward Tufte's brilliant book set including The visual display of quantitative information and Envisioning information are required reading for anyone designing visual displays.
  5. Direct manipulation was coined by Ben Shneiderman, and some of the discussion of it is inspired from his paper Direct manipulation: A step beyond programming languages, IEEE Computer 1992, 16(8).
  6. The reading Tools and Techniques for Creative Design supplied pretty well all the raw material for creativity.