Controlled Study: Mouse Typing on Various Keyboards

HCILectures.ControlledStudyKeyboardComparison History

Show minor edits - Show changes to markup

Added lines 10-16:

Learning Overview

Learning Goals. This assignment is a hands-on exercise on running a controlled study. Its immediate purpose is to give you experience conducting a controlled study, in generating data, in analyzing that data via statistical methods, in interpretting the results, and critiquing the experiment.

Learning Scenario. You will assume your team is working for a company that is considering various on-screen keyboard designs, where a person types with their mouse. You will use each other as subjects, where you will each run at least one or more subjects (usually each other). You will contribute the data to a pool of data provided by other class members. You will independantly analyze the collective results. Details are below, with software and additional material provided in the appendices listed below.

Deliverables. Your will deliver a technical report that includes the background and methodology (most which is already provided), your particular observations and results, the collective observations and results, a description of your statistical analysis and findings, an interpretation of these results, some design recommendations of what keyboard should be used and the perceived benefits/drawbacks, and a critique of the study method.

Deleted lines 26-32:

Learning Overview

Learning Goals. This assignment is a hands-on exercise on running a controlled study. Its immediate purpose is to give you experience conducting a controlled study, in generating data, in analyzing that data via statistical methods, in interpretting the results, and critiquing the experiment.

Learning Scenario. You will assume your team is working for a company that is considering various on-screen keyboard designs, where a person types with their mouse. You will use each other as subjects, where you will each run at least one or more subjects (usually each other). You will contribute the data to a pool of data provided by other class members. You will independantly analyze the collective results. Details are below, with software and additional material provided in the appendices above.

Deliverables. Your will deliver a technical report that includes the background and methodology (most which is already provided), your particular observations and results, the collective observations and results, a description of your statistical analysis and findings, an interpretation of these results, some design recommendations of what keyboard should be used and the perceived benefits/drawbacks, and a critique of the study method.

Added lines 20-26:

Learning Overview

Learning Goals. This assignment is a hands-on exercise on running a controlled study. Its immediate purpose is to give you experience conducting a controlled study, in generating data, in analyzing that data via statistical methods, in interpretting the results, and critiquing the experiment.

Learning Scenario. You will assume your team is working for a company that is considering various on-screen keyboard designs, where a person types with their mouse. You will use each other as subjects, where you will each run at least one or more subjects (usually each other). You will contribute the data to a pool of data provided by other class members. You will independantly analyze the collective results. Details are below, with software and additional material provided in the appendices above.

Deliverables. Your will deliver a technical report that includes the background and methodology (most which is already provided), your particular observations and results, the collective observations and results, a description of your statistical analysis and findings, an interpretation of these results, some design recommendations of what keyboard should be used and the perceived benefits/drawbacks, and a critique of the study method.

Added lines 4-9:

Note

A few things are left to do with this material

  • proof-read the text
  • test the task along with instructions
  • include a page describing how to do the Anova and Correlation/Regression
Changed lines 70-71 from:

Method. Each subject goes through the following steps, detailed further in Appendix 1.

to:

Method.

Each subject goes through the following steps, detailed further in Appendix 1.

Changed line 97 from:

Results

to:

Results: the Data

Changed lines 96-98 from:

3. RESULTS Each subject will be identified by a unique number. You should collect your numeric and qualitative results on the worksheets as given in Appendix 5. Your numeric data should be recorded in EXACTLY the format described on the worksheet. The data collected from all calss mates will then be returned to you.

to:

Results

Each subject will be identified by a unique number. You should collect your numeric and qualitative results on the worksheets provided in the Appendix. Your numeric data should be recorded in EXACTLY the format described on the worksheet. The data collected from all calss mates will then be returned to you.

Added lines 4-13:
Deleted lines 62-63:
Changed line 67 from:

The various appendices describes how to download and use the custom software that presents a soft keyboard. Appendices also include instructions for downloading

to:

The various appendices listed on describes how to download and use the custom software that presents a soft keyboard. Appendices also include instructions for downloading

Changed lines 47-55 from:

Null Hypotheses 1a (simpler): There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard.

Null Hypotheses 1b(full version): Null Hypotheses 1a (simple form): There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard, regardless of people's touch typing abilities as measured in characters per second.

Null Hypotheses 2: People's typing speeds when touch-typing is not correlated with mouse-typing speeds, where both are measured in characters per second.

to:

Null Hypotheses 1a (simpler): There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard.

Null Hypotheses 1b (complete: There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard, regardless of people's touch typing abilities as measured in characters per second.

Null Hypotheses 2: People's typing speeds when touch-typing is not correlated with mouse-typing speeds, where both are measured in characters per second.

Changed lines 29-33 from:

The various keyboards are displayed below. All keyboards can be done in two available sizes, as illustrated with the Qwerty keyboard.

Qwerty (Large)

to:

The various keyboards are displayed below. All keyboards can be displayed in a large and small size (not shown).

Changed lines 75-76 from:
  • When subjects are ready, the experimenter presses the Start button, and the subject types the

paragraph.

to:
  • When subjects are ready, the experimenter presses the Start button, and the subject types the paragraph.
Added lines 92-106:

3. RESULTS Each subject will be identified by a unique number. You should collect your numeric and qualitative results on the worksheets as given in Appendix 5. Your numeric data should be recorded in EXACTLY the format described on the worksheet. The data collected from all calss mates will then be returned to you.

For hypothesis 1, calculate an ANOVA contrasting the typing speed of layouts and sizes. Do the same with the error rate.

For hypothesis 2, do a correlation / regression analysis, showing the line of best fit that contrasts subjects' typing speed on the conventional typing test (the X-axis) with their typing speed on each of the other "soft" keyboards (the Y-axis).

Your write up should include a table of summary statistics and descriptions of these result.

Analysis / Conclusion

An experimental report typically follows the style specified in the accompanying document "How to structure reports on experiments in human-computer interaction". This assignment specification already gives material for the introduction, description of the experiment, and expectations in the results section. It would also include a related work section, and it would be up to you to investigate other keyboard studies. It is up to you to interpret what the results mean. You should discuss this with other experimentors, and think about it. Think about the impact your findings would have to the application constraints described in this scenario, the general use of soft keyboards, the implications to other practitioners, your critical reflections, and a research agenda of future study and issues. Use both the qualitative and quantitative findings of your study, as well as prior research on keyboards.

Your conclusion should make a recommendation and should justify your results. Include your rough data and worksheets in an appendix. This should include the work sheets where you recorded preand post-experiment answers to questions, subject's comments, etc. You would not normally include this in a paper, but you should always maintain this as an archive that you can use if you have to refer back to your data and its analysis.

Changed lines 8-9 from:

Your company produces a kiosk-based system, where all input is expected to be done with a mouse. Typically, clients will buy this system for walk-up and use situations. Clients will install applications on this system. Customers, will typically walk up and use the application in the kiosk . For most client applications, customers will use the mouse to do some simple selections. However, various applications also ask clients to enter some text, ranging from a few characters to several lines of text. Consequently, your company has decided to create an on-screen keyboard that allows people to mouse-type.

to:

Your company produces a kiosk-based system, where all input is expected to be done with a mouse. Typically, clients will buy this system for walk-up and use situations. Clients will install applications on this system. Customers, will typically walk up and use the application in the kiosk. For most client applications, customers will use the mouse to do some simple selections. However, various applications also ask clients to enter some text, ranging from a few characters to several lines of text. Consequently, your company has decided to create an on-screen keyboard that allows people to mouse-type.

Changed lines 47-51 from:

You will be collecting data and comparing through statistics the mouse-typing speeds on the various layouts and sizes. The null hypothesis are listed below. Simpler hypotheses can be stated, although these limit the scope of the study (and typically require different statistical analysis methods).

Null Hypotheses 1: There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard.

to:

You will be collecting data and comparing through statistics the mouse-typing speeds on the various layouts and sizes. The null hypothesis are listed below. Note that two forms are provided for Hypothesis 1, where the second form includes people's typing abilities.

Null Hypotheses 1a (simpler): There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard.

Null Hypotheses 1b(full version): Null Hypotheses 1a (simple form): There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard, regardless of people's touch typing abilities as measured in characters per second.

Added lines 57-58:
Changed lines 60-65 from:
 Subjects are members of your group, all University of Calgary students. You may use other subjects as well, but make sure they have equivalent computing and typing experience. A broad range of typing speeds are preferred.

2.3 Materials. Appendix 2 describes how to get, test and use the custom software that presents a soft keyboard showing only the letter keys and a space bar on a screen. Other appendices provide instructions for downloading software, instructions to subjects, text for copy typing, consent forms, questionnaires, and individual work sheets.

to:

Normally, subjects with a broad range of typing skills would be selected from the public at large to fit the demographic profile. In this case, your subjects are chosen from members of your class, all University of Calgary students.

Materials.

The various appendices describes how to download and use the custom software that presents a soft keyboard. Appendices also include instructions for downloading software, instructions to subjects, text for copy typing, consent forms, questionnaires, and individual work sheets.

Method. Each subject goes through the following steps, detailed further in Appendix 1.

  1. Prolog:
    • The subject is asked to sign a consent form (Appendix 3).
    • The experiment is described to them (Appendix 1).
    • They answer a pre-test questionnaire (Appendix 4).
    • They are given additional instructions as they perform the task (Appendix 1).
  2. Trial 1-2: Touch-typing speed with physical keyboard
    • Subjects are given two English paragraphs to touch type using the physical keyboard (Appendix 2).
    • When subjects are ready, the experimenter presses the Start button, and the subject types the

paragraph.

  • Immediately after the subject finishes the task, the experimenter presses the Stop button
  • The process repeats itself with a new paragraph.
  1. Trials 3-14: Typing speed with the six keyboards
    • The experimenter selects one of the keyboards and sizes(order to be determined by your Instructor) and describes it to the subject.
    • The subject is given an English sentence to mouse type (Appendix 2). After reading it, the subject presses the Start button and enters the sentence. When complete, the subject presses the Stop button. Note: make sure they press this button, as timing stops when it is pressed.
    • The process is repeated on the same keyboard layout with two more sentences.
    • After the keyboard is tested, ask subjects what they thought about the layout.
    • The process is repeated with the other keyboard layout.
  2. Post-session
    • Ask subjects what they thought about all keyboards, and have them state which one they preferred (Appendix
    • Ask them for any other comments.
    • Debrief them. Explain your motivation if they wish (for example, expectations about keyboard layouts)
Changed lines 24-25 from:

Essentially, your company disagrees as to which is best. The interaction design group believes that the qwerty keyboard is best overall: they are familiar to experienced typists, and inexperienced typists will likely be equally slow regardless of the layout. The marketing department believes that the alphabetic keyboard is better for inexperienced typists, and that experienced typists can do fine on them because mouse-typing is different from touch typing. The company president is neutral, but is interested in knowing how either keyboard compares to a worse-case situation, i.e., where keys are randomly placed on a keyboard (that is, does layout even matter in mouse typing?). The hardware group believes that people will be much faster on large keyboards than small keyboards, and is arguing that the small screen product should thus be abandoned. The sales group says that size doesn't matter, and that it would be a mistake to stop small screen production as their clients slightly prefer small screens as they are cheaper.

to:

Essentially, your company disagrees as to which is best. The interaction design group believes that the qwerty keyboard is best overall: they are familiar to experienced typists, and inexperienced typists will likely be equally slow regardless of the layout. The marketing department believes that the alphabetic keyboard is better for inexperienced typists, and that experienced typists can do fine on them because mouse-typing is different from touch typing. The company president is neutral, but is interested in knowing how either keyboard compares to a worse-case situation, i.e., where keys are randomly placed on a keyboard (that is, does layout even matter in mouse typing?), and if touch-typing speed is even relevant. The hardware group believes that people will be much faster on large keyboards than small keyboards, and is arguing that the small screen product should thus be abandoned. The sales group says that size doesn't matter, and that it would be a mistake to stop small screen production as their clients slightly prefer small screens as they are cheaper.

Added lines 42-60:

Description of the Experiment

Hypothesis.

You will be collecting data and comparing through statistics the mouse-typing speeds on the various layouts and sizes. The null hypothesis are listed below. Simpler hypotheses can be stated, although these limit the scope of the study (and typically require different statistical analysis methods).

Null Hypotheses 1: There is no difference between people's mouse-typing ability as measured in speed (characters per second) and error rate (ratio of extra characters typed over the minimum required) when using a qwerty, alphabetic, or random layouts on two different sizes of a simulated keyboard.

Null Hypotheses 2: People's typing speeds when touch-typing is not correlated with mouse-typing speeds, where both are measured in characters per second.

Subjects.

 Subjects are members of your group, all University of Calgary students. You may use other subjects as well, but make sure they have equivalent computing and typing experience. A broad range of typing speeds are preferred.

2.3 Materials. Appendix 2 describes how to get, test and use the custom software that presents a soft keyboard showing only the letter keys and a space bar on a screen. Other appendices provide instructions for downloading software, instructions to subjects, text for copy typing, consent forms, questionnaires, and individual work sheets.

Added lines 32-33:

Qwerty (Large)

Added lines 35-36:

Qwerty (Small)

Added lines 38-39:

Alphabetic (Small)

Added line 41:

Random (Small) - the actual keys shown vary

Added lines 27-34:

The Keyboards

The various keyboards are displayed below. All keyboards can be done in two available sizes, as illustrated with the Qwerty keyboard.

Attach:ControlledStudyKeyboardComparisionQwerty.png Attach:ControlledStudyKeyboardComparisionQwertySmall.png Attach:ControlledStudyKeyboardComparisionAlphabetic.png Attach:ControlledStudyKeyboardComparisionRandom.png

Added lines 4-26:

Introduction

The Problem

Your company produces a kiosk-based system, where all input is expected to be done with a mouse. Typically, clients will buy this system for walk-up and use situations. Clients will install applications on this system. Customers, will typically walk up and use the application in the kiosk . For most client applications, customers will use the mouse to do some simple selections. However, various applications also ask clients to enter some text, ranging from a few characters to several lines of text. Consequently, your company has decided to create an on-screen keyboard that allows people to mouse-type.

Some constraints are already known:

  • typing of numbers and text will be done by displaying a simulated keyboard on the screen, and by selecting keys with the mouse (i.e., mouse-typing);
  • no hardware keyboard or keypad will be available
  • end-customers may vary considerably in standard typing abilities;
  • people need to be able to enter their text as quickly as possible (as otherwise line-ups may occur)
  • errors in typed text should be minimized (as entered information often goes into a database and can be critical)
  • the hardware product is available in two screen sizes, although the larger one is more costly.

Several on-screen keyboard layout designs are currently being considered. - a standard Qwerty keyboard - an Alphabetic keyboard - a Random keyboard. - all the above come in two possible keyboard sizes

Essentially, your company disagrees as to which is best. The interaction design group believes that the qwerty keyboard is best overall: they are familiar to experienced typists, and inexperienced typists will likely be equally slow regardless of the layout. The marketing department believes that the alphabetic keyboard is better for inexperienced typists, and that experienced typists can do fine on them because mouse-typing is different from touch typing. The company president is neutral, but is interested in knowing how either keyboard compares to a worse-case situation, i.e., where keys are randomly placed on a keyboard (that is, does layout even matter in mouse typing?). The hardware group believes that people will be much faster on large keyboards than small keyboards, and is arguing that the small screen product should thus be abandoned. The sales group says that size doesn't matter, and that it would be a mistake to stop small screen production as their clients slightly prefer small screens as they are cheaper.

You are in charge of finding out which one is best, and making a recommendation. You have decide to run an controlled experiment comparing typing speed and error rate on these keyboards, and will use the results to make recommendations about which layout should be used in the product.

Changed line 1 from:

(:title Controlled Study: A Comparison of Keyboards :)

to:

(:title Controlled Study: Mouse Typing on Various Keyboards :)

Changed lines 1-3 from:

(:title Controlled Study: A Comparison of Keyboards :)

to:

(:title Controlled Study: A Comparison of Keyboards :) Back to assignment overview

Changed line 1 from:

(:title Controlled Study: A Comparison of Keyboard :)

to:

(:title Controlled Study: A Comparison of Keyboards :)

Changed line 1 from:

(:title Controlled Study: A Keyboard Comparison :)

to:

(:title Controlled Study: A Comparison of Keyboard :)

Added line 1:

(:title Controlled Study: A Keyboard Comparison :)