Notes for Teaching Assistants Lab 3
Topics covered are:
- Presentations of Assignment 1 prototype and walkthrough summaries by students
- Fnal questions on Assignment 1 write-up
- Assignment 2: Usability study:
- schedule of dates and labs (see lab schedule)
- brief discussion of assignment
- go over handout
- what they are supposed to do
- what is in the write-up
Assignment 1 Presentations
Most of this lab time will be taken up by student presentations. Stress to students
that you will *not* be evaluating (at least not in a big way) the quality of the
interface, although you may make some comments about it ie, I have not yet taught them
much about actual interface design! I am more interested in how they use the prototyping
methods and the insights gained in the walkthroughs. Tell them that they will have a
chance to reflect on their designs and change them when they begin the project.
In the presentations:
- The most important thing for you to do is to catch students who are far off base, and
set them onto the right track. If you are unsure, send them to me (but not too many,
- they should be using either storyboards or Pictive techniques or some variation. See how
well they use these techniques to show their interface.
- In particular,
- does the prototype show the main interface style/representation?
- does it show how the interface progresses through at least one of the core tasks?
- does it give you a good feel of the interface style?
- have they indicated other interface representations (ask them what other interface
representations they considered, if any)
- For the task walkthrough summaries
- were you convinced that they actually did the walkthrough, and that the insights they
gained from it are reasonable?
- are they concentrating on minor or major problems (they should be considering only major
- Ask them if they have any questions on the write-up for Assignment 1
- Remind them that late assignments will not be accepted, that they must be well written,e
- Tell them to look at the grading sheet for an indication of what we expect
Assignment 2: Usability assignment-introduction
- Stress that many usability problems can be fixed at little or no extra costs in
- Stress that usability studies are a cheap, effective way of discovering design flaws
- For the assignment, briefly describe the system to be evaluated and its purpose (you
should try it out yourself)
- Mention that each group should use at least 3-4 subjects: themselves (excluding
experimenter), and preferably one or two others. The more the better.
- another way to do it is to use a ''partner group'' as a subject base-> some tasks
will differ from what yours are, and you will be able to discuss the results together.
- each group select an experimenter
- the experimenter will prepare the example tasks ahead of time (preferably by discovering
what real tasks are by interviewing system users)
- the experimenter will also prepare a short pre-test questionnaire that solicits people's
prior experience with computers and the chosen system (why is this important?) and the
user's expectations of their experience. Similarly, the experimenter will prepare a
post-test questionnaire. Make sure that they justify their choice of questions i.e. that
they think the results will give them meaningful information.
- run each subject through pre-test questionnaire
- when all is ready, the experimenter will run the tests according to the assignment spec:
- silent observer:
- subject 1
- observer & subject not allowed to speak to each other
- observer takes notes of subjectís behavior, especially where breakdowns occur
- think aloud
- subject 2
- subject asked to talk aloud explain what they are doing/thinking elaborate on
- observer takes notes
- constructive interaction/co-discovery learning
- subjects 3 & 4 (or subjects 1 & 2 on a more complex task)
- both work together on task, with one being the system driver
- observer again takes notes
- administer post-test questionnaire
- interview the subject, using questions based on observations and questionnaire
Go over the assignment sheet specs. Some added comments for different sections are
- Perspective: Pretend that you are a product evaluation team for the company that
developed the system, and that you are looking for major flaws to repair for a new
- Section 4: Interpretation system strengths & weaknesses
- Identify common and important problems & strengths
- break into categories
- should be more than a checklist of all the problems seen!
- Try to generalize problems when necessary, although you can use examples to highlight
- The intent is to highlight what you think are the important observations
- Section 5: Suggested improvements
- Pick at least 5 important changes you would make
- Must be restricted to low-cost changes e.g., same hardware and interface style
- Appendix 1: Comparison of different techniques
- Assume we know what the techniques are
- Summarize your experience with each technique comparison, ease of use, information
obtained, advantages, etc
- How would you do this if you were going to do it again? What would you keep? Throw away?
- Appendix 2: Raw data
Last updated April 1998, by Saul Greenberg