Location-dependant Object controller

Return to Idea Sketches

This is an idea I had several years ago, but had trouble finding a student to take it on.

Basic premise.

A location-dependan object controller (yes, its a horrible name)is an untethered mobile device with a reasonable size screen (e.g., tablet, or a PDA). If a person approaches a 'controllable' object (it could be digital or real world), the controller senses that object (e.g., through ARTags, RFID, or bar codes) and links to a view of that object. The person can then see information related to that object, and perhaps even control its properties.

If the sensing system gives you proximity information (e.g., ARTags, Vicon), then the view of that information as well as the degree of control would be a function of proximity.


Many devices in our real world are small - too small to provide a reasonable view into the information it may contain. As well, interactions with that device are often compromised due to cost, size etc (think of, for example, your digital watch, your home thermostat, an ambient display, etc.). Why not bring larger screens with better viewing and interaction facilities to the device?


Ambient displays

The purpose behind most ambient displays is to provide awareness information about something. These devices are usually located in some context amenable to that dispaly e.g., so it is easily seen as people walk by. The catch is that it is sometimes difficult to move from awareness to exploration and even interaction with that information. For example, imagine we had a figurine (connected to an Instant Messenging system) that lights up to different degrees when a particular person is online. While we know that that person is there, actually moving into conversation requires a much more complex interface. Instead, we would approach the figuring with our controller, and the controller would immediately display more information about that person, the ability to chat with them, and also the ability to reassign who that figurine represents. A version of this is presented in a video (see Harrison's work; look up reference).

Location-dependand displays

Similarly, Katherine Elliot's location dependant devices can be viewed in further detail. Currently, one needs to swipe an RFID card to assign a function to these devices. But our controller could do this in a richer way, and perhaps give people further options as to how information (and what information) is assigned to these devices.

Universal controller

There are already many devices that we control by remotes: televisions, cd players, ipods, dvd players, etc. The display could fuse these into a single control by being aware of what appliances are in a room. Brad Myers did some work on Universal controllers.


If you approach a vending machine, you can buy things through your controller. This is already done with cell phones (often very badly!); the interface is often terrible due to the cell phone interface (many menus / buttons).

Home inspection

We have many warning lights that tell us when things go right or wrong. These are often presented as crypted LEDs (a green flashing may mean ok, but time for a checkup) or uninformative messages in cars (in my Suburu, there is a 'check engine' message; however, I don't know if its a serious problem or not). The controller can present this information in a much more meaningful way, and perhaps give me some options of what to do about it (e.g., connect to google and find local service people)

Consumer information

Given a product, find out more information about it .e.g, when shopping (Marc Smith did this; get reference).

and on and on.


Brad Myers describes an XML-based method to show and communicate relevant information between devices, where the device tries to generate an interface from it. An early version of VNC (Get Reference) actually had devices use the VNC protocol to generate a richer interface. Perhaps a better option is to have each device be associated with a web handle, where it publishes / subscribes to information in that handle (e.g., shared dictionary). Associated with that handle is also a program - perhaps a java applet - that can be downloaded automatically to the tablet.

We would implement location awareness through ARTags (as with HOme Window), with RFID tags with the reader taped onto the tablet (Nic did this with his Shared Phidgets project), or even with the Vicon if we want a fine degree of information and control.


  • flavours of this have been done before, but I am not sure if its been done as comprehensively as suggested here. Need to gather the background research. Rob Diaz started this once...
  • Some devices we would want to control do not have network capability, e.g., my home thermostat, my stove, my car. We would have to simulate this (or make our own appliances using phidgets )