CPSC 681 - Topic: Researching and Presenting a Methodology

CPSC681.Project History

Hide minor edits - Show changes to markup

Changed lines 4-112 from:

Worth: 65% of your grade

Project type

Most students do a project involving an evaluation. However, there are actually several types of projects you can do. If you want to do something that is not an evaluation, talk to me as soon as possible.

  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience planning, running, debugging, analyzing and interpretting the results of an evaluative study. You will:
    • define an interesting (but modest) problem doable within the course time frame
    • learn how to apply a particular methodological technique to that problem
    • write an ethics application
    • plan a study
    • conduct an initial pilot study and a more substantial pilot study
    • analyze and interpret the results
    • critically appraise the limitations of the study
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  2. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. You will:
    • define a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • define how the team would learn and/or use it
    • develop a design rationale and a design process for the usability engineering process
    • deploy the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraise what worked and what didn't, and suggesting modifications
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  3. Craft a software tool that supports a methodological process. You will:
    • define an interesting portion of a methodology that is amenable to software support
    • define the audience and when and where it would be useful
    • develop a design rationale and a design process to bring to bear on that methodology
    • iteratively design and implement the system
    • conduct a simple study using your software tool to see how well it performs in practice
    • critically appraise the strengths limitations of the system
    • present your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

Individual projects are allowable, but in practice the ideal team size is two to three people due to the amount of work involved. Your group can include people not in the course, e.g., faculty / senior student mentors (with permission of the instructor). While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups.

Independent work

This is an independent project, with work done outside of class time. Each group is responsible for carrying out all aspects of the project (e.g., defining the project, logistics, finding subjects if necessary, booking rooms and equipment). While I will guide you through some of the work if needed, I will not hold your hand.

Example Milestones for Evaluations

I will give you a schedule containing required milestones and approximately when they are due. Because most projects are based upon some type of user observation, the milestones are crafted around that. For other projects, I will tailor milestones to fit.

1. Problem statement and suggested methodological approach

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to one or more of:

  • your thesis work or that of a class member or colleague,
  • a replication of a published study,
  • systems being worked on at your job site (if you are an employee, but be careful of Intellectual Property),
  • an on-going research project (or related side-project) in your research lab or
  • something that I give you to do

Deliverable. A concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Note: Problems selected must be amenable to one of the methodological approaches discussed in class. This is somewhat backwards; while the class philosophy is to choose a methodology that is appropriate to a problem and the needs of the researcher, we may have to craft the problem so that it is appropriate to a selected methodology, the time you have, and the limited resources available to you!

2. Ethics review application

Research involving evaluations with human subjects require an ethics application, which you will fill out. Because this assignment is for pedagogical learning vs. actual research, I will not require you to submit this application to the ethics board.

Note: Without ethics approval, you cannot use your pilot results for research purposes, i.e., this is strictly a pedagogical learning exercise. You can submit it if you wish, but that is totally up to you. Approval delays will likely conflict with course schedule demands, so you should be prepared to do at least part of your project as a learning exercise until approval is granted.

Important links:

  • University of Calgary’s website for Research Ethics
  • Do I Need to Apply For Ethics Approval?
  • Information to help applicants to fill out the application form
  • Ethics Application Form that you should fill out
  • Example template for a consent form

Deliverable: The deliverable is the completed ethics application. Note that you will have to have part of the next deliverable completed to do this. Note that the application will include part of the 'Detailed Description' below.

3. Detailed description of problem and methodological setup

See How to Structure Reports on Experiments in Human Computer Interaction.

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format. As described in How to Structure Reports on Experiments in Human Computer Interaction, it will likely contain:

  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis

4. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

Deliverable: The deliverable is a modification and extension of the previous one:

  • Title, authors, abstract, etc
  • Motivation, background, and detailed problem statement,
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Summary of changes to problem statement/methodological approach

5. Running the study

Most students end up doing a limited pilot i.e., a scaled down study of a few subjects (typically your class colleagues). Regardless, you will probably have to:

  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results

6. Final report and presentation

Deliverables:

  • Professional quality (ACM CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 15 minute class presentation plus 5 minutes questions in professional conference format
to:

(:title CPSC 681 - Topic: Researching and Presenting a Methodology :) back to CPSC 681

Go to this page on The New Web Site

This page has been replace.

Changed lines 73-74 from:
to:
Changed lines 72-73 from:
  • Ethics Application Form that you should fill out
to:
  • Ethics Application Form that you should fill out
Changed lines 72-73 from:
  • Ethics Application Form that you should fill out
to:
  • Ethics Application Form that you should fill out
Changed lines 72-73 from:
  • Ethics Application Form that you should fill out
to:
  • Ethics Application Form that you should fill out
Changed lines 72-73 from:
  • Ethics Application Form that you should fill out
to:
  • Ethics Application Form that you should fill out
Changed line 70 from:
  • Do I Need to Apply For Ethics Approval?
to:
  • Do I Need to Apply For Ethics Approval?
Changed lines 71-73 from:
  • Information to help applicants to fill out the application form
  • Ethics Application Form that you should fill out
to:
  • Information to help applicants to fill out the application form
  • Ethics Application Form that you should fill out
Changed lines 70-71 from:
to:
  • Do I Need to Apply For Ethics Approval?
  • Information to help applicants to fill out the application form
  • Ethics Application Form that you should fill out
Added lines 68-71:

Important links:

  • University of Calgary’s website for Research Ethics
Changed line 75 from:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format. As described in How to Structure Reports on Experiments in Human Computer Interaction, it will contain:

to:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format. As described in How to Structure Reports on Experiments in Human Computer Interaction, it will likely contain:

Changed line 84 from:

Deliverable: The deliverable is a modification and extension of the previous one:

to:

Deliverable: The deliverable is a modification and extension of the previous one:

Changed lines 64-65 from:

If you do an evaluation requiring human subjects, you must fill out and submit an ethics application as fast as possible.

to:

Research involving evaluations with human subjects require an ethics application, which you will fill out. Because this assignment is for pedagogical learning vs. actual research, I will not require you to submit this application to the ethics board.

Note: Without ethics approval, you cannot use your pilot results for research purposes, i.e., this is strictly a pedagogical learning exercise. You can submit it if you wish, but that is totally up to you. Approval delays will likely conflict with course schedule demands, so you should be prepared to do at least part of your project as a learning exercise until approval is granted.

Deleted lines 100-101:

Note: Without ethics approval, you cannot use your pilot results for research purposes, i.e., this is strictly a pedagogical learning xercise.

Added line 70:
Changed line 73 from:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain:

to:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format. As described in How to Structure Reports on Experiments in Human Computer Interaction, it will contain:

Changed lines 82-83 from:

Deliverable: The deliverable is a variation of the previous one:

  • Detailed problem statement, motivation, and background
to:

Deliverable: The deliverable is a modification and extension of the previous one:

  • Title, authors, abstract, etc
  • Motivation, background, and detailed problem statement,
Changed lines 89-90 from:
  • Changes to problem statement/methodological approach
to:
  • Summary of changes to problem statement/methodological approach
Changed line 92 from:

Most students end up doing a limited pilot i.e., a scaled down study. If you can do a full study, you will probably have to:

to:

Most students end up doing a limited pilot i.e., a scaled down study of a few subjects (typically your class colleagues). Regardless, you will probably have to:

Added lines 99-100:

Note: Without ethics approval, you cannot use your pilot results for research purposes, i.e., this is strictly a pedagogical learning xercise.

Changed line 103 from:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
to:
  • Professional quality (ACM CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work.
Changed line 105 from:
  • 15 + 5 minute class presentation in conference format
to:
  • 15 minute class presentation plus 5 minutes questions in professional conference format
Changed lines 7-38 from:

Most students do an evaluation. However, there are actually several types of projects you can do. If you want to do something that is not an evaluation, talk to me as soon as possible.

  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience
    • defining an interesting (but modest) problem,
    • learning how to apply a particular methodological technique to that problem
    • writing an ethics application
    • planning a study
    • conducting both pilot and the main study
    • analyzing and interpreting the results
    • critically appraising the limitations of the study
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
  2. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. The experience you will gain is:
    • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • defining how the team would learn and/or use it
    • developing a design rationale and a design process for the usability engineering process
    • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraising what worked and what didn't, and suggesting modifications
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
  3. Craft a software tool that supports a methodological process. The experience you will gain is:
    • defining an interesting portion of a methodology that is amenable to software support
    • defining the audience and when and where it would be useful
    • developing a design rationale and a design process to bring to bear on that methodology
    • actual iterative system design
    • conducting a study using your software tool to see how well it performs in practice
    • critically appraising the limitations of the system
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
to:

Most students do a project involving an evaluation. However, there are actually several types of projects you can do. If you want to do something that is not an evaluation, talk to me as soon as possible.

  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience planning, running, debugging, analyzing and interpretting the results of an evaluative study. You will:
    • define an interesting (but modest) problem doable within the course time frame
    • learn how to apply a particular methodological technique to that problem
    • write an ethics application
    • plan a study
    • conduct an initial pilot study and a more substantial pilot study
    • analyze and interpret the results
    • critically appraise the limitations of the study
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  2. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. You will:
    • define a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • define how the team would learn and/or use it
    • develop a design rationale and a design process for the usability engineering process
    • deploy the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraise what worked and what didn't, and suggesting modifications
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  3. Craft a software tool that supports a methodological process. You will:
    • define an interesting portion of a methodology that is amenable to software support
    • define the audience and when and where it would be useful
    • develop a design rationale and a design process to bring to bear on that methodology
    • iteratively design and implement the system
    • conduct a simple study using your software tool to see how well it performs in practice
    • critically appraise the strengths limitations of the system
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
Changed lines 41-42 from:

The ideal team size is two to three people, and can include people not in the course. While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups. Each group is responsible for carrying out all aspects of the project (e.g., finding subjects if necessary, booking rooms and equipment); while I will guide you through some of the work if needed, I will not hold your hand.

to:

Individual projects are allowable, but in practice the ideal team size is two to three people due to the amount of work involved. Your group can include people not in the course, e.g., faculty / senior student mentors (with permission of the instructor). While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups.

Independent work

This is an independent project, with work done outside of class time. Each group is responsible for carrying out all aspects of the project (e.g., defining the project, logistics, finding subjects if necessary, booking rooms and equipment). While I will guide you through some of the work if needed, I will not hold your hand.

Changed lines 48-51 from:

I will give you a schedule containing required milestones and approximately when they are due. I suspect that most projects will be based upon some type of user observation, and the milestones are crafted around that. For other projects, I will tailor milestones to fit.

The work will have to be done outside of class time.

to:

I will give you a schedule containing required milestones and approximately when they are due. Because most projects are based upon some type of user observation, the milestones are crafted around that. For other projects, I will tailor milestones to fit.

Changed line 51 from:

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to:

to:

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to one or more of:

Changed lines 53-54 from:
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
to:
  • a replication of a published study,
  • systems being worked on at your job site (if you are an employee, but be careful of Intellectual Property),
  • an on-going research project (or related side-project) in your research lab or
Changed lines 64-67 from:

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible.

Deliverable: The deliverable is the completed ethics application. Note that you will have to have part of the next deliverable completed to do this.

to:

If you do an evaluation requiring human subjects, you must fill out and submit an ethics application as fast as possible.

Deliverable: The deliverable is the completed ethics application. Note that you will have to have part of the next deliverable completed to do this. Note that the application will include part of the 'Detailed Description' below.

Changed line 69 from:

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)\\

to:
August 27, 2009, at 05:25 PM by 24.64.89.150 -
Changed lines 54-58 from:
  • something that I give you to do

Deliverable. A concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

to:
  • something that I give you to do

Deliverable. A concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Changed lines 68-69 from:

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

to:

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

Changed lines 77-78 from:

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

to:

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

August 27, 2009, at 05:24 PM by 24.64.89.150 -
Added line 56:
Added line 61:
August 27, 2009, at 05:24 PM by 24.64.89.150 -
Added lines 20-28:
  1. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. The experience you will gain is:
    • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • defining how the team would learn and/or use it
    • developing a design rationale and a design process for the usability engineering process
    • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraising what worked and what didn't, and suggesting modifications
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
Deleted lines 38-46:
  1. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. The experience you will gain is:
    • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • defining how the team would learn and/or use it
    • developing a design rationale and a design process for the usability engineering process
    • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraising what worked and what didn't, and suggesting modifications
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
August 27, 2009, at 05:23 PM by 24.64.89.150 -
Deleted line 1:
Changed lines 3-5 from:

Project type

While most students do an evaluation, there are several types of projects you can do. If you want to do something not on this list, talk to me as soon as possible.

to:

Worth: 65% of your grade

Project type

Most students do an evaluation. However, there are actually several types of projects you can do. If you want to do something that is not an evaluation, talk to me as soon as possible.

August 27, 2009, at 05:22 PM by 24.64.89.150 -
Changed line 47 from:

Problem statement and suggested methodological approach

to:

1. Problem statement and suggested methodological approach

Changed line 58 from:
  1. Ethics review application\\
to:

2. Ethics review application

Changed line 63 from:
  1. Detailed description of problem and methodological setup\\
to:

3. Detailed description of problem and methodological setup

Changed line 73 from:
  1. Pilot study\\
to:

4. Pilot study

Changed line 84 from:
  1. Running the study\\
to:

5. Running the study

Changed line 92 from:
  1. Final report and presentation\\
to:

6. Final report and presentation

August 27, 2009, at 05:21 PM by 24.64.89.150 -
Changed line 47 from:
  1. Problem statement and suggested methodological approach\\
to:

Problem statement and suggested methodological approach

August 27, 2009, at 05:20 PM by 24.64.89.150 -
Changed lines 53-55 from:

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.
Note: Problems selected must be amenable to one of the methodological approaches discussed in class. This is somewhat backwards; while the class philosophy is to choose a methodology that is appropriate to a problem and the needs of the researcher, we may have to craft the problem so that it is appropriate to a selected methodology, the time you have, and the limited resources available to you!

to:

Deliverable. A concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.
Note: Problems selected must be amenable to one of the methodological approaches discussed in class. This is somewhat backwards; while the class philosophy is to choose a methodology that is appropriate to a problem and the needs of the researcher, we may have to craft the problem so that it is appropriate to a selected methodology, the time you have, and the limited resources available to you!

Changed lines 59-61 from:

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.
Deliverable: The deliverable is the completed application. Note that you will have to have part of the next deliverable completed to do this.

to:

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible.

Deliverable: The deliverable is the completed ethics application. Note that you will have to have part of the next deliverable completed to do this.

Changed lines 66-67 from:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain:

to:

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain:

Changed lines 75-76 from:

Deliverable: The deliverable is a variation of the previous one:

to:

Deliverable: The deliverable is a variation of the previous one:

Changed line 93 from:

Deliverables:

to:

Deliverables:

Deleted lines 96-97:

Last updated Fall 2004

August 27, 2009, at 05:18 PM by 24.64.89.150 -
Changed lines 44-45 from:
  • Most of the work will have to be done outside of class time.
to:

The work will have to be done outside of class time.

August 27, 2009, at 05:18 PM by 24.64.89.150 -
Changed lines 18-19 from:
  1. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:
to:
  1. Craft a software tool that supports a methodological process. The experience you will gain is:
Changed lines 28-29 from:
  1. Define and introduce a usability engineering process to your interface development team.
to:
  1. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. The experience you will gain is:
August 27, 2009, at 05:16 PM by 24.64.89.150 -
Changed line 11 from:
  • writing an ethics application
to:
  • writing an ethics application
August 27, 2009, at 05:16 PM by 24.64.89.150 -
Added line 11:
  • writing an ethics application
Changed line 87 from:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
to:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
Changed lines 89-90 from:
  • 25 + 5 minute class presentation in conference format
to:
  • 15 + 5 minute class presentation in conference format
Changed line 2 from:
to:
November 29, 2006, at 11:23 AM by 136.159.7.242 -
Changed line 51 from:

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

to:

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.\\

Changed line 59 from:

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)

to:

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)\\

November 29, 2006, at 11:22 AM by 136.159.7.242 -
Changed line 45 from:
  1. Problem statement and suggested methodological approach
to:
  1. Problem statement and suggested methodological approach\\
Changed line 50 from:
  • something that I give you to do
to:
  • something that I give you to do\\
Changed lines 54-55 from:
  1. Ethics review application

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.

to:
  1. Ethics review application
    If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.\\
Changed line 58 from:
  1. Detailed description of problem and methodological setup
to:
  1. Detailed description of problem and methodological setup\\
Changed lines 60-61 from:

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect. Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain

to:

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.
Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain:

Changed lines 67-68 from:
  1. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

to:
  1. Pilot study
    Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.\\
Changed line 77 from:
  1. Running the study
to:
  1. Running the study\\
Changed line 85 from:
  1. Final report and presentation
to:
  1. Final report and presentation\\
November 29, 2006, at 10:40 AM by 136.159.7.242 -
Deleted line 45:
Deleted line 54:
Deleted line 58:
Deleted line 67:
Deleted line 77:
Deleted line 85:
November 29, 2006, at 10:39 AM by 136.159.7.242 -
Changed lines 48-51 from:
  • your thesis work or that of a class member or colleague,
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
  • something that I give you to do
to:
  • your thesis work or that of a class member or colleague,
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
  • something that I give you to do
Changed lines 65-69 from:
  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis
to:
  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis
Changed lines 74-80 from:
  • Detailed problem statement, motivation, and background
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Changes to problem statement/methodological approach
to:
  • Detailed problem statement, motivation, and background
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Changes to problem statement/methodological approach
Changed lines 84-89 from:
  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results
to:
  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results
Changed lines 93-96 from:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 25 + 5 minute class presentation in conference format
to:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 25 + 5 minute class presentation in conference format
November 29, 2006, at 10:38 AM by 136.159.7.242 -
Deleted line 42:
Changed lines 45-46 from:
  1. Problem statement and suggested methodological approach
to:
  1. Problem statement and suggested methodological approach
Changed lines 48-51 from:
  • your thesis work or that of a class member or colleague,
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
  • something that I give you to do
to:
  • your thesis work or that of a class member or colleague,
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
  • something that I give you to do
Changed lines 54-56 from:
  1. Ethics review application
to:
  1. Ethics review application
Changed lines 59-61 from:
  1. Detailed description of problem and methodological setup
to:
  1. Detailed description of problem and methodological setup
Changed lines 65-69 from:
  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis
  1. Pilot study
to:
  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis
  1. Pilot study
Changed lines 74-80 from:
  • Detailed problem statement, motivation, and background
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Changes to problem statement/methodological approach
  1. Running the study
to:
  • Detailed problem statement, motivation, and background
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Changes to problem statement/methodological approach
  1. Running the study
Changed lines 84-89 from:
  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results
  1. Final report and presentation
to:
  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results
  1. Final report and presentation
Changed lines 93-96 from:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 25 + 5 minute class presentation in conference format
to:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 25 + 5 minute class presentation in conference format
November 29, 2006, at 10:20 AM by 136.159.7.242 -
Changed lines 7-8 from:
  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience
to:
  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience
Changed lines 16-18 from:
  1. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:
to:
  1. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:
Changed lines 26-28 from:
  1. Define and introduce a usability engineering process to your interface development team.
to:
  1. Define and introduce a usability engineering process to your interface development team.
November 29, 2006, at 09:25 AM by 136.159.7.242 -
Changed lines 4-5 from:

Project type

to:

Project type

Changed lines 31-32 from:

Project Teams

to:

Project Teams

Changed lines 35-36 from:

Example Milestones for Evaluations

to:

Example Milestones for Evaluations

November 29, 2006, at 09:24 AM by 136.159.7.242 -
Changed lines 5-6 from:

Project type

to:

Project type

Deleted lines 7-8:
Changed lines 9-15 from:
  • defining an interesting (but modest) problem,
  • learning how to apply a particular methodological technique to that problem
  • planning a study
  • conducting both pilot and the main study
  • analyzing and interpreting the results
  • critically appraising the limitations of the study
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
to:
  • defining an interesting (but modest) problem,
  • learning how to apply a particular methodological technique to that problem
  • planning a study
  • conducting both pilot and the main study
  • analyzing and interpreting the results
  • critically appraising the limitations of the study
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
Changed lines 17-23 from:
  • defining an interesting portion of a methodology that is amenable to software support
  • defining the audience and when and where it would be useful
  • developing a design rationale and a design process to bring to bear on that methodology
  • actual iterative system design
  • conducting a study using your software tool to see how well it performs in practice
  • critically appraising the limitations of the system
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
to:
  • defining an interesting portion of a methodology that is amenable to software support
  • defining the audience and when and where it would be useful
  • developing a design rationale and a design process to bring to bear on that methodology
  • actual iterative system design
  • conducting a study using your software tool to see how well it performs in practice
  • critically appraising the limitations of the system
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
Changed lines 25-33 from:
  • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
  • defining how the team would learn and/or use it
  • developing a design rationale and a design process for the usability engineering process
  • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
  • critically appraising what worked and what didn't, and suggesting modifications
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

to:
  • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
  • defining how the team would learn and/or use it
  • developing a design rationale and a design process for the usability engineering process
  • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
  • critically appraising what worked and what didn't, and suggesting modifications
  • presenting your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

Changed lines 36-37 from:

Example Milestones for Evaluations

to:

Example Milestones for Evaluations

November 29, 2006, at 09:21 AM by 136.159.7.242 -
Deleted lines 4-5:
Changed line 1 from:

(: title CPSC 681 - Project :)

to:

(:title CPSC 681 - Project :)

Added lines 1-2:

(: title CPSC 681 - Project :)

Changed lines 5-6 from:

CPSC 681 - Project

to:
November 28, 2006, at 03:15 PM by 136.159.7.242 -
Changed line 80 from:

6. Final report and presentation

to:
  1. Final report and presentation
November 28, 2006, at 03:15 PM by 136.159.7.242 -
Added line 38:
Changed lines 50-51 from:
  • something that I give you to do
    Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.\\
to:
  • something that I give you to do

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Changed line 54 from:

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.\\

to:

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.

Changed lines 57-58 from:

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)
You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.\\

to:

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex) You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

Changed line 65 from:

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.\\

to:

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

November 28, 2006, at 03:14 PM by 136.159.7.242 -
Changed lines 74-80 from:
    * book rooms if needed
    * prepare any necessary equipment and/or software
    * solicit and schedule subjects
    * run subjects and collect data/observations
    * analyze the data and interpret the results
to:
  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results
Deleted line 79:
Changed lines 81-86 from:
    * Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
    * Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
    * 25 + 5 minute class presentation in conference format

Last updated Fall 2004

to:
  • Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 25 + 5 minute class presentation in conference format

Last updated Fall 2004

November 28, 2006, at 03:11 PM by 136.159.7.242 -
Changed lines 42-45 from:
    * Most of the work will have to be done outside of class time.

1. Problem statement and suggested methodological approach

to:
  • Most of the work will have to be done outside of class time.
  1. Problem statement and suggested methodological approach
Changed lines 46-55 from:
    * your thesis work or that of a class member or colleague,
    * systems being worked on at your job site (if you are working),
    * an on-going research project in your research lab or
    * something that I give you to do

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

to:
  • your thesis work or that of a class member or colleague,
  • systems being worked on at your job site (if you are working),
  • an on-going research project in your research lab or
  • something that I give you to do
    Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.\\
Changed lines 52-55 from:

2. Ethics review application

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.

to:
  1. Ethics review application

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.\\

Changed lines 55-60 from:

3. Detailed description of problem and methodological setup

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

to:
  1. Detailed description of problem and methodological setup

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)
You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.\\

Changed lines 59-79 from:
    * Detailed problem statement, motivation, and background
    * Methodology
    * Example of your expected data / results (made up)
    * Example analysis

4. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

Deliverable: The deliverable is a variation of the previous one

    * Detailed problem statement, motivation, and background
    * Methodology used for pilot study
    * Data and results observed
    * Initial analysis of results
    * Expected trends
    * Changes to problem statement/methodological approach

5. Running the study

to:
  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis
  1. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.
Deliverable: The deliverable is a variation of the previous one:

  • Detailed problem statement, motivation, and background
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Changes to problem statement/methodological approach
  1. Running the study
November 28, 2006, at 03:08 PM by 136.159.7.242 -
Changed lines 11-20 from:
   1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience

    * defining an interesting (but modest) problem,
    * learning how to apply a particular methodological technique to that problem
    * planning a study
    * conducting both pilot and the main study
    * analyzing and interpreting the results
    * critically appraising the limitations of the study
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.
to:
  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience
    • defining an interesting (but modest) problem,
    • learning how to apply a particular methodological technique to that problem
    • planning a study
    • conducting both pilot and the main study
    • analyzing and interpreting the results
    • critically appraising the limitations of the study
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
  2. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:
    • defining an interesting portion of a methodology that is amenable to software support
    • defining the audience and when and where it would be useful
    • developing a design rationale and a design process to bring to bear on that methodology
    • actual iterative system design
    • conducting a study using your software tool to see how well it performs in practice
    • critically appraising the limitations of the system
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.
  3. Define and introduce a usability engineering process to your interface development team.
    • defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • defining how the team would learn and/or use it
    • developing a design rationale and a design process for the usability engineering process
    • deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraising what worked and what didn't, and suggesting modifications
    • presenting your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

The ideal team size is two to three people, and can include people not in the course. While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups. Each group is responsible for carrying out all aspects of the project (e.g., finding subjects if necessary, booking rooms and equipment); while I will guide you through some of the work if needed, I will not hold your hand. Example Milestones for Evaluations

I will give you a schedule containing required milestones and approximately when they are due. I suspect that most projects will be based upon some type of user observation, and the milestones are crafted around that. For other projects, I will tailor milestones to fit.

    * Most of the work will have to be done outside of class time.

1. Problem statement and suggested methodological approach

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to:

    * your thesis work or that of a class member or colleague,
    * systems being worked on at your job site (if you are working),
    * an on-going research project in your research lab or
    * something that I give you to do

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Deleted lines 56-98:
   2. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:

    * defining an interesting portion of a methodology that is amenable to software support
    * defining the audience and when and where it would be useful
    * developing a design rationale and a design process to bring to bear on that methodology
    * actual iterative system design
    * conducting a study using your software tool to see how well it performs in practice
    * critically appraising the limitations of the system
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.



   3. Define and  introduce a usability engineering process to your interface development team.

    * defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    * defining how the team would learn and/or use it
    * developing a design rationale and a design process for the usability engineering process
    * deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    * critically appraising what worked and what didn't, and suggesting modifications
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

The ideal team size is two to three people, and can include people not in the course. While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups. Each group is responsible for carrying out all aspects of the project (e.g., finding subjects if necessary, booking rooms and equipment); while I will guide you through some of the work if needed, I will not hold your hand. Example Milestones for Evaluations

I will give you a schedule containing required milestones and approximately when they are due. I suspect that most projects will be based upon some type of user observation, and the milestones are crafted around that. For other projects, I will tailor milestones to fit.

    * Most of the work will have to be done outside of class time.

1. Problem statement and suggested methodological approach

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to:

    * your thesis work or that of a class member or colleague,
    * systems being worked on at your job site (if you are working),
    * an on-going research project in your research lab or
    * something that I give you to do

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

November 28, 2006, at 03:05 PM by 136.159.7.242 -
Added lines 1-116:

back to CPSC 681

CPSC 681 - Project

Project type

While most students do an evaluation, there are several types of projects you can do. If you want to do something not on this list, talk to me as soon as possible.

   1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience

    * defining an interesting (but modest) problem,
    * learning how to apply a particular methodological technique to that problem
    * planning a study
    * conducting both pilot and the main study
    * analyzing and interpreting the results
    * critically appraising the limitations of the study
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.



   2. Craft a software tool that supports a methodological process (e.g., on-line support of heuristic evaluation). The experience you will gain is:

    * defining an interesting portion of a methodology that is amenable to software support
    * defining the audience and when and where it would be useful
    * developing a design rationale and a design process to bring to bear on that methodology
    * actual iterative system design
    * conducting a study using your software tool to see how well it performs in practice
    * critically appraising the limitations of the system
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.



   3. Define and  introduce a usability engineering process to your interface development team.

    * defining a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    * defining how the team would learn and/or use it
    * developing a design rationale and a design process for the usability engineering process
    * deploying the usability engineering process to evaluate a (small) project that arises from a real situation
    * critically appraising what worked and what didn't, and suggesting modifications
    * presenting your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

The ideal team size is two to three people, and can include people not in the course. While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups. Each group is responsible for carrying out all aspects of the project (e.g., finding subjects if necessary, booking rooms and equipment); while I will guide you through some of the work if needed, I will not hold your hand. Example Milestones for Evaluations

I will give you a schedule containing required milestones and approximately when they are due. I suspect that most projects will be based upon some type of user observation, and the milestones are crafted around that. For other projects, I will tailor milestones to fit.

    * Most of the work will have to be done outside of class time.

1. Problem statement and suggested methodological approach

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to:

    * your thesis work or that of a class member or colleague,
    * systems being worked on at your job site (if you are working),
    * an on-going research project in your research lab or
    * something that I give you to do

Deliverable. Concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Note: Problems selected must be amenable to one of the methodological approaches discussed in class. This is somewhat backwards; while the class philosophy is to choose a methodology that is appropriate to a problem and the needs of the researcher, we may have to craft the problem so that it is appropriate to a selected methodology, the time you have, and the limited resources available to you! 2. Ethics review application

If you do an evaluation requiring human subjects, you must submit an ethics application as fast as possible. See the readings for how this is done.

Deliverable: The deliverable is the completed application. Note that you will have to have part of the next deliverable completed to do this. 3. Detailed description of problem and methodological setup

See How to Structure Reports on Experiments in Human Computer Interaction (doc) or (latex)

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format and will contain

    * Detailed problem statement, motivation, and background
    * Methodology
    * Example of your expected data / results (made up)
    * Example analysis

4. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

Deliverable: The deliverable is a variation of the previous one

    * Detailed problem statement, motivation, and background
    * Methodology used for pilot study
    * Data and results observed
    * Initial analysis of results
    * Expected trends
    * Changes to problem statement/methodological approach

5. Running the study

Most students end up doing a limited pilot i.e., a scaled down study. If you can do a full study, you will probably have to:

    * book rooms if needed
    * prepare any necessary equipment and/or software
    * solicit and schedule subjects
    * run subjects and collect data/observations
    * analyze the data and interpret the results

6. Final report and presentation

Deliverables:

    * Professional quality (CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work. The CHI format style sheet is included here.
    * Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
    * 25 + 5 minute class presentation in conference format

Last updated Fall 2004