Tangible & Physical HCI

CPSC 599.88 (Undergraduate) / CPSC 601.7 (Graduate)

Instructor: Lora Oehlberg

Teaching Assistant: Kurtis Danyluk

Office Hours: By appointment, MS 280H or MS 142 (Garage 142)

Lecture: Tu/Th | 15:30-16:45 | MS 211

Grading

Schedule

Reading Responses

Throughout the course, we will have sessions discussing specific research papers from human-computer interaction research that address Tangible User Interface theory and systems.

You should be able to access most readings through the University of Calgary library – if you are using campus internet, it should automatically authorize you to access all research papers on the ACM DIgital Library. For off campus access, you should set up EZ Proxy as a bookmarklet on your browser.

To ensure that all students are prepared to participate in the in-class discussion, you will submit a reading response before the beginning of class on the day of the discussion. Each reading response should:

• In one or two sentences, summarize the authors' main contribution of the paper.
• Tell me what you personally found the most interesting or useful from this paper.
• Sketch (or describe) two new ideas for Tangible User Interfaces that emerge from your reading of this paper.

Submit reading responses on D2L. Go to the Discussions page, enter the forum for Reading Responses, and go to the topic on the reading for that week. Add a new thread with your response; you will be able to view others' reponses after you have posted your own thread/response.

List of Readings

• Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI ’97). ACM, New York, NY, USA, 234–241.
• Scott R. Klemmer, Björn Hartmann, and Leila Takayama. 2006. How bodies matter: five themes for interaction design. In Proceedings of the 6th conference on Designing Interactive Systems (DIS ’06). ACM, New York, NY, USA, 140–149.
• Joseph Malloch and Marcelo M. Wanderley. 2007. The T-Stick: from musical interface to musical instrument. In Proceedings of the 7th international conference on New Interfaces for Musical Expression (NIME ’07). ACM, New York, NY, USA, 66–70.
• Mathieu Le Goc, Charles Perin, Sean Follmer, Jean-Daniel Fekete and Pierre Dragicevic, Dynamic Composite Data Physicalization Using Wheeled Micro-Robots". In IEEE Transactions on Visualization and Computer Graphics, vol. 25, no. 1, pp. 737-747, Jan. 2019.
• Annemiek Veldhuis, Rong-Hao Liang, and Tilde Bekker. 2020. CoDa: Collaborative Data Interpretation Through an Interactive Tangible Scatterplot. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’20). Association for Computing Machinery, New York, NY, USA, 323–336.

Assignments

The guidelines below apply to all assignments.

Course Materials

To complete all assignments you will need to obtain an Arduino Starter Kit. We recommend ordering it from Solarbotics or visiting Active Tech; you may also find kits available on Amazon. If you already have an Arduino or Arduino-compatible platform at home (e.g., SEEEduino, Freeduino, Lilypad, etc.), you may also use this and acquire your own basic electronics kit (with a breadboard, jumper cables, resistors, sensors, and output).

We are using the Arduino platform because it is a well-established microcontroller platform with many existing online tutorials, documentation, and libraries. Keep track of your use of these resources, and please cite them in your documentation.

I strongly recommend bringing your Arduino materials with you to lecture. We will often have "open work time" towards the end of class where you can work on your project, ask questions, or get debugging help.

If you have difficulty obtaining an Arduino and electronics starter kit, or if you would rather develop on an alternative platform, please contact the instructor as soon as possible.

Sourcing Electronic Components

• Engineering Stores. On-Campus, in Shulich. CNRL Engineering Complex, ENC 025
• Solarbotics / ActiveTech. Local electronics shop.
• Digikey (online, ships from Vancouver)
• RobotShop (online, ships from QC)
• Elmwood Electronics (online, ships from Toronto, ON)
• Sparkfun (online, international shipping to Canada from US)
• Adafruit (online, ships from US)

"Creative Surprise"

I expect students in this course to exercise their creativity on all assignments — going beyond satisfying specifications to consider how their design can differentiate their project from their peers, take a risk, or apply something beyond the scope of the course to their project. The one constraint on “Creative Surprise” is to continuously stretch beyond your comfort zone. I will looking for how you continually push boundaries throughout the semester.

That said, your "creative surprise" does not literally need to surprise your instructor or TA. Please spoil the surprise to ask for help on your projects. Asking for help will ensure that your conceptual creativity can find a way to be implemented into reality.

In-Class Critique

Part of your assignment or project grade will be participation in an in-class critique session. At the start of class, all students will have 10 minutes to set up their projects at a demo station. The whole class will walk around the room to see each others' demos, ask questions, and provide feedback.

Your demo should include the following:

• Introduce your project. Give it a name, since everyone has an "Assignment Three". Yours is unique.
• Elevator pitch. Summarize the concept of your project in 1-2 minutes.
• Demonstrate Interaction. Provide a live demo of your project in action. You may need to set up a "demo mode" to your project specifically for the in-class demo, particularly if the interaction is intended to occur over a long time period.
• If time, explain how it works. You may be cut off, since Q&A may include specific questions on implementation.

If you need any special accomodation to demo your project (e.g., help physically getting it into the room, turning off the smoke detectors so your demo does not trigger a building evacuation, etc.), please reach out to the course instructors ASAP.

Documentation

You must document all your assignments or projects such that someone else could clearly read through it, understand what you did, and recreate your project. Project documentation involves five key components:

• A title and brief summary (abstract). This is the "elevator pitch" of what you have created. You should be able to succinctly summarize what you did, and capture what makes your project different, insightful, or unique. This written abstract should be under 300 words.
• Visuals that capture your project. Please include both (a) visuals that capture your design process (concept sketches, photos of construction in-progress, any early prototypes) and (b) one or several "glamour shots" of your project, well-lit on a neutral background (think National Geographic-style photography).
• A video demonstration of your project. Video is the best way to capture the experience of interaction with new technologies or systems. While a simple video may capture your project clearly, some simple video editing (adding titles, including multiple shots, etc.) may help you more clearly communicate what your project does, and how it works. The video should be no longer than 2 minutes; it should include subtitles, intertitles, or a voice-over to describe the system; it should include a title card with: Project Title, Your Name, Course Title, Date. You may choose to host your videos on YouTube, or host them locally on your webpage.
• Your Arduino Code. Your code should be clean and well-commented. Your code should be able to run self-sufficiently from within a single directory. Please submit your code as a .zip file that can be downloaded, or as a link to GitHub.
• Circuit Schematic. This course will require all students to document the physical schematic of their circuits. You can learn electronics notation and hand-sketch this on graph paper, or document it electronically. We recommend creating a Fritzing diagram (schematic view, not the more illustrative 'breadboard view') that represents all components and circuit connections. You can use the appropriate Fritzing components to reflect the microcontroller and electronic hardware used for your specific project.
• Reference Materials. It is important to acknowledge any influence from external projects or learning resources that helped you in your project. Please cite any libraries or online documentation/tutorials used to inform or implement your project.

Submitting Project Documentation

1. Your name, and the title of your project
2. Link to your Portfolio Website. You should send me the link to the specific page that documents your project.
3. PDF or Screenshot of your Webpage. So we know that the verision that we look at is the one you intend to be evaluated.
4. Quick Reflection. Write a quick note that addresses the following three questions:
   1. How much time did you spend on the assignment outside of class?
   2. What did you learn during this project?
   3. What advice would you give to a future student working on this assignment?

With permission, I will list students' portfolio websites on this webpage.

Here are some sources of inspiration of what quality online project documentation can look like.

Assignment 0: Class Startup

Due 21 January, in class. C/NC.
3% of Final Grade

Purpose

Checking that all students are set up to complete the assigments and projects required in the course.

The Brief

There's several things you need to do to get set up for this course:

• Complete the Introductory Survey! This helps us tailor the course to your needs, and gives us a heads-up on what equipment we need.
• Obtain a microprocessor (e.g., Arduino) and basic electronics kit. I recommend ordering an Arduino Kit from Solarbotics. Solarbotics merged with ActiveTech recently; I recommend making an in-person visit to become familiar with what they stock. This may come in handy for future assignments.
• Set up Arduino IDE on your laptop or tablet
• Set up EZ Proxy to access research readings off-campus
• Download Fritzing (optional)
• Peruse the Arduino Reference Documentation and Tutorials

Assignment 1: Arduino Hello World

Due 28 January, in class.
7%% of Final Grade

Purpose

1. introduce the very basics of building programmable circuits.
2. introduce electronics documentation (demo video, circuit diagrams or Fritzing)
3. introduce the overall format and expectations for assignments.

The Brief

This term, we will be using the Arduino Microprocessor. The Arduino microprocessor connects to your laptop via USB, and have pin-outs that connect to circuits that contain the sensors and actuators that constitute an interface. It's a popular platform, and a good foundation for future physical computing projects.

In this assignment, you will create a simple circuit – an LED that is controlled by a button. (Easy enough.) However, you must also include a Creative Surprise" to differentiate your project from your peers (I won't be surprised by a verbatim implementation of an existing tutorial).

Please bring your project to class on 28 January and be prepared to demonstrate your concept in a 2-3 minute presentation.

Successful Projects Will Include:

1. an LED (basic output)
2. a Button (basic input)
3. Creative Surprise!
4. Complete documentation

Assignment 2: Physical Input

Due 13 February, in class.
25% of Final Grade

Students will complete this assignment in pairs. Your partner for this assignment must be different from your partner for Assignment 3 and the Final Project. When you have identified your project partner, sign-up as a group for this Assignment on D2L.

Purpose

The purpose of this assignment is to:

1. encourage exploration of various physical inputs
2. control non-physical output (sound)
3. initial exploration of embodiment

The Brief

Build a new musical instrument or expressive musical interface.

You should consider the holistic experience of someone performing the instrument, and/or the experience of the audience listening to a performance. I encourage you to explore alternative mappings and combine input techniques as a way of controlling sound output.

Remember: make me an expressive instrument. Your instrument should react to a performer, not play pre-programmed sounds (an MP3 player), and not work through a pre-determined sequence (a playlist). It should be malleable and controllable, and not dependent on computational 'randomness' for its expressive character.

You should consider the role of embodiment in your design — physical computing is not simply about electronic hardware and the software that defines its behavior, but also how it fits into a physical embodiment that people interact with. Do not submit a “squid”; try going beyond a rectangular cardboard box and consider form.

As with all assignments, go beyond the brief. Surprise me.

Successful Projects Will Include:

1. an innovative physical input
2. sound output that maps to physical input
3. a physical embodiment for that instrument
4. Creative Surprise!
5. Complete documentation

Assignment 3: Physical Output

Due 10 March, in class.
25% of Final Grade

Students will complete this assignment in pairs. Your partner for this assignment must be different from your partner for Assignment 2 and the Final Project. When you have identified your project partner, sign-up as a group for this Assignment on D2L.

Purpose

1. explore types of physical, tangible output
2. integrate data streams with Arduino
3. Continued exploration with embodiment and physical metaphor

The Brief

Create an interactive physical visualization — an object whose physical behavior or dynamic state represents data. Your data physicalization should represent personal data, corresponding to yourself or another specific person.

Consider how this interactive physical visualization might have a role in someone’s everyday life. In your documentation, tell the story of where the data comes from, and why someone would care about or interact with the physicalization.

You may find that one of the most natural mappings for an interactive physical visualization is time-based/temporal data. However, you are free to select any personal dataset that you would like to work with.

If your physical visualization reflects a slow-moving live data stream, be sure to include a 'demo' mode for your project that demonstrates how it reacts to data, based on simulated data.

The physical embodiment of the visualization should thematically or metaphorically map to the meaning of the data itself.

As with all assignments, go beyond the brief. Surprise me.

Successful Projects Will Include

1. an innovative physical output
2. real-world dataset reflected as physical output
3. a physical embodiment that metaphorically connects to the data itself
4. Creative Surprise!
5. Complete documentation

Course Project

In-Class Presentations on 7 April and 9 April, in class
Documentation due on 14 April, by 3:30 p.m. (begininng of class)

Purpose

1. Combine physical input with physical output
2. Push the boundaries of electronics implementation
3. Push the boundaries of physical embodiment
4. Communicate an aesthetic idea through a tangible interface

The Brief

Create a wearable "fashionable" interface that communicates or reifies the concept of hygge. While hygge tends to be a uniquely Danish concept; you are welcome to translate "cozy" into your (Canadian) cultural context. Please choose a person, context, or setting to help ground your hygge wearable and experience.

As with everything in this course, go beyond the brief. Surprise me.

Mid-Project Feedback Critique

Your Mid-Project Feedback Critique will be C/NC, based on participation. You should come to the feedback critique well-prepared to present your ideas in progress, and ready and open for eliciting feedback from your peers ("I like..." / "I wish..." / "What if...?").

On D2L, submit four slides (title slide, 3 slides with one concept visual per slide) by Monday 23 March at noon.

Final Project Presentation

Final Project Presentations will be in-class on 7 April and 9 April. Sign-ups for presentation time slots will be on 26 March, either in-class or online (more detail provided via email). The overall format for the Final Project presentations will be similar to the format of assignment demo and critique sessions.

Final Project Documentnation

As with other assignments, follow the documentation guidelines and create an online portfolio page about that project. Documentation will be due at class time, on the final day of class, 14 April 2020.

Successful Projects Will Include

1. A compelling story that links the person wearing the artifact, their context or situation, and how the artifact behaves in response to interaction (mapping)
2. Physical input
3. Physical output, and/or other (e.g., sound, light) forms of output
4. Wearable physical embodiment that connects to the aesthetic
5. An aesthetic of hygge
6. Creative Surprise!
7. Complete Documentation

Resources

• Kobakant "How to Get What You Want", sewable sensors and electronic components.
• Calgary MakeFashion
• Introduction to Lilypad Arduino

For Graduate Students

Research Presentations

Graduate students will be required to give a research presentation during the course lecture, and lead an in-class discussion around a research paper. These sessions will be scheduled with the course instructor at the beginning of the term.

A note on Design Briefs & Thesis Topics

All assignments and projects will be completed by graduate students individually.

With consent of instructor, graduate students may develop design briefs for course assignments that address their thesis topic. However, assignments will still be evaluated based on the original intended purpose of that assignment (input, output, synthesis). Undergraduate students must adhere to the original design brief.

Assignment & Project Documentation

Instead of a portfolio website, graduate students should submit assignment and project documentation in the form of an ACM SIGCHI Extended Abstract, using the templates from CHI 2020.

Student Portfolios

• Bader Abdulwaseem
• Katie Tieu
• Rahmanta Satriana