The Product. Air Canada Online is a real commercial web-based system (http://www.aircanada.ca/). The site offers many features. You and your team will evaluate how 'Guests' use the Web interface to Air Canada Online site (you will ignore the portion of the site dealing with Login as Member). We will pretend that the Vice President in charge of the site wants to know if web-literate young people, such as university students, can successfully accomplish a few typical travel tasks. Your job is to find major flaws with this portion of the site by observing people do these tasks, and then to recommend ways to fix them. If there are problems, the Vice President may authorize changes to the system in upcoming versions.
Selecting Tasks. It is up to you to decided what set of typical tasks should be given to the users. The assignment sheet has a section that indicates how you can go about this, and you are already familiar with task descriptions from Assignment 1. As well, the experimenter should try the system ahead of time, becoming as familiar with it as possible. The experimenter should come up with at least six other reasonable tasks to give to subjects, preferably more. A good task one that is likely to be used by many end-users. Tasks should also be selected to investigate different (but still important or heavily used) parts of the system functionality.
To get you going, I've included a few sample tasks below. Notice that they are phrased as directions that will be given to the user.
Preparing equipment. You can have people use browsers from their own account, or you can have a browser set up in your account. If you are reusing a browser (e.g., in your login account) clear the browser's cache before each session, as you don't want the link highlights to give the user clues of where to go. Depending on how you run subjects, you may want to have a browser (e.g., Netscape or Internet Explorer) up and ready at the Air Canada Online home page (http://www.aircanada.ca/), although I strongly recommend that you have people start cold for at least the first task (e.g., how to find the site.).
A few precautionary notes.
Questionnaires. Administer the pre-test questionnaire. Questions must at least probe for people's experience with the computer they are using, the web browser they are using, travel reservation systems (both paper and electronic), the Air Canada site, and how often they make their own air travel arrangements. You may also want to ask them if they are familiar with web-based search systems.
At the end of the test, administer the post-test questionnaires. These should include questions that ask people how satisfied they are with the system (e.g., "I would rather check ticket fares with the existing Air Canada site over phoning a travel agent" (Strongly agree, agree, neither agree nor disagree, disagree, strongly disagree).
Instructions. Administer the usability instructions to subjects, as indicated in the handout.
Initial conceptual model. Note: the reason you are doing this is to see what initial conceptual model people have of the system, based upon their prior experiences and their interpretation of the visuals on the screen. You are looking for places where the model is incorrect. Start doing this as soon as they get to the main screen of Air Canada Online.
Task 1. Finding the site.You are interested in booking a ticket for a flight on Air Canada. You recall reading in the newspaper that they had a web site that let you look at plane schedules and book tickets on-line. Using whatever means you want, find the page that lets you do this and display it in your web browser..
Reason for choosing this task. No matter how good the web site is, it is only useful if people can actually find it. This task will see what people do to try to find the site, as well as how successful they are at it (e.g., if it is easily found via their chosen search engine, or by typing in a suspected URL, etc.)
Task 2. Checking a flight arrival. You have a friend arriving on a flight tonight from Toronto. Unfortunately, you've lost the paper with all the details on it. You recall she is flying Air Canada, and that her flight arrives after 9pm. Try and find the flight.
Reason for choosing this task. This is a typical (although less frequent) task, where people have a fuzzy request to make.
Task 3. Finding a cheap flight on a simple trip. Go to the Air Canada Online home page (http://www.aircanada.ca/). You want to check costs of a flight from Calgary to Montreal, where you leave around breakfast time on Wednesday morning this week and arrive back after supper Friday evening. You definitely want to book a cheap flight, but you also prefer direct flights over those with stopovers (unless there is a big cost difference). If the ticket cost seems expensive, you may be willing to stay over Saturday night for a cheaper ticket (are they) as long as you can get back early on Sunday.
Its up to you to discover which tickets best suits your needs.
Reason for choosing this task. This is a simple but routine request. The customer knows their travel constraints, and the travel arrangements are straight forward (eg, most likely as direct flights with no stop-overs, with some flexibility in travel schedule if cost differences are large). Also, this tests whether the person can navigate through the system to the appropriate pages.
Task 4. A complex task. Find a cheap itinerary that lets you travel from Calgary to Boston, where you will stay in Boston for the weekend, and then travel from Boston to Miami on the Monday. Return to Calgary on Friday.
Reason for choosing this task. Many people have quite complex travel considerations. The one above is not unusual.
Other tasks you could develop. The experimenter should explore the system. If the experimenter suspects certain problems, the experimenter can develop a task that exercises that part of the system.
The developing conceptual model. When complete, repeat the exercise the subject did at the beginning i.e., have them explain what each interface component does in each main screen. Has their conceptual model changed through their experiential learning? Is their model correct?