CPSC 681 - Project

back to CPSC 681

Worth: 65% of your grade

Project type

Most students do a project involving an evaluation. However, there are actually several types of projects you can do. If you want to do something that is not an evaluation, talk to me as soon as possible.

  1. Apply methodological techniques to evaluate situations in human computer interaction. The purpose of this type of project is to give you experience planning, running, debugging, analyzing and interpretting the results of an evaluative study. You will:
    • define an interesting (but modest) problem doable within the course time frame
    • learn how to apply a particular methodological technique to that problem
    • write an ethics application
    • plan a study
    • conduct an initial pilot study and a more substantial pilot study
    • analyze and interpret the results
    • critically appraise the limitations of the study
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  2. Define and introduce a usability engineering process to your interface development team. This is typically appropriate for students who are working full time as a software developer. You will:
    • define a realistic usability engineering / educational process that is amenable to your organization, budget, and work
    • define how the team would learn and/or use it
    • develop a design rationale and a design process for the usability engineering process
    • deploy the usability engineering process to evaluate a (small) project that arises from a real situation
    • critically appraise what worked and what didn't, and suggesting modifications
    • present your work in a professional manner, both through writing a technical report and through oral presentation.
  3. Craft a software tool that supports a methodological process. You will:
    • define an interesting portion of a methodology that is amenable to software support
    • define the audience and when and where it would be useful
    • develop a design rationale and a design process to bring to bear on that methodology
    • iteratively design and implement the system
    • conduct a simple study using your software tool to see how well it performs in practice
    • critically appraise the strengths limitations of the system
    • present your work in a professional manner, both through writing a technical report and through oral presentation.

Project Teams

Individual projects are allowable, but in practice the ideal team size is two to three people due to the amount of work involved. Your group can include people not in the course, e.g., faculty / senior student mentors (with permission of the instructor). While these teams can be self-selecting, I have the final say. I will likely insist that stronger (experienced) team members include novice researchers within their groups.

Independent work

This is an independent project, with work done outside of class time. Each group is responsible for carrying out all aspects of the project (e.g., defining the project, logistics, finding subjects if necessary, booking rooms and equipment). While I will guide you through some of the work if needed, I will not hold your hand.

Example Milestones for Evaluations

I will give you a schedule containing required milestones and approximately when they are due. Because most projects are based upon some type of user observation, the milestones are crafted around that. For other projects, I will tailor milestones to fit.

1. Problem statement and suggested methodological approach

The problem must be one relevant to human-computer interaction and of interest to you and your team. Ideally, it will be related to one or more of:

  • your thesis work or that of a class member or colleague,
  • a replication of a published study,
  • systems being worked on at your job site (if you are an employee, but be careful of Intellectual Property),
  • an on-going research project (or related side-project) in your research lab or
  • something that I give you to do

Deliverable. A concise problem statement, why it is interesting to you and your team, what methodology you will bring to bear on this problem, and why you feel that methodology is appropriate.

Note: Problems selected must be amenable to one of the methodological approaches discussed in class. This is somewhat backwards; while the class philosophy is to choose a methodology that is appropriate to a problem and the needs of the researcher, we may have to craft the problem so that it is appropriate to a selected methodology, the time you have, and the limited resources available to you!

2. Ethics review application

Research involving evaluations with human subjects require an ethics application, which you will fill out. Because this assignment is for pedagogical learning vs. actual research, I will not require you to submit this application to the ethics board.

Note: Without ethics approval, you cannot use your pilot results for research purposes, i.e., this is strictly a pedagogical learning exercise. You can submit it if you wish, but that is totally up to you. Approval delays will likely conflict with course schedule demands, so you should be prepared to do at least part of your project as a learning exercise until approval is granted.

Important links:

Deliverable: The deliverable is the completed ethics application. Note that you will have to have part of the next deliverable completed to do this. Note that the application will include part of the 'Detailed Description' below.

3. Detailed description of problem and methodological setup

See How to Structure Reports on Experiments in Human Computer Interaction.

You should describe the problem in detail and say exactly how your group will conduct the study. Methodological steps should be detailed enough so that someone else could actually run the experiment! You should also describe the data you will be collecting, how it will be analyzed, and what results you expect.

Deliverable: The deliverable is a draft of the first 3rd of your final report. It will be in ACM CHI format. As described in How to Structure Reports on Experiments in Human Computer Interaction, it will likely contain:

  • Detailed problem statement, motivation, and background
  • Methodology
  • Example of your expected data / results (made up)
  • Example analysis

4. Pilot study

Experiments, like computer programs, are full of bugs. Save yourself time and effort by running yourselves (and your class mates) through your procedure. It would be nice if you could practice your analysis on the data produced by these subjects. This is a great time to catch the bugs and get rid of them. It is also a good time to get a sense of what the real results will be like, and perhaps to consider what changes should be made to the problem statement and methodological approach.

Deliverable: The deliverable is a modification and extension of the previous one:

  • Title, authors, abstract, etc
  • Motivation, background, and detailed problem statement,
  • Methodology used for pilot study
  • Data and results observed
  • Initial analysis of results
  • Expected trends
  • Summary of changes to problem statement/methodological approach

5. Running the study

Most students end up doing a limited pilot i.e., a scaled down study of a few subjects (typically your class colleagues). Regardless, you will probably have to:

  • book rooms if needed
  • prepare any necessary equipment and/or software
  • solicit and schedule subjects
  • run subjects and collect data/observations
  • analyze the data and interpret the results

6. Final report and presentation

Deliverables:

  • Professional quality (ACM CHI format) report that integrates all the above, and that includes background, conclusions, and suggestions for further work.
  • Archival and well-structured binder / CD that details all your research (this will include everything mentioned above)
  • 15 minute class presentation plus 5 minutes questions in professional conference format