So now that UIST 2011 is officially over, I can post about the work that I did at Autodesk Research in January of 2010! I’m super excited to finally get to talk about all my hard work and the awesome project (Medusa) that I got to work on. For now, I just want to share the video and a brief description of what I did (it was 4 months of work after all!). The full paper outlining Medusa (‘Medusa: a proximity-aware multi-touch tabletop’) can be found here.

Quick summary:

So in short, Medusa is a Microsoft Surface that has been instrumented with 138 proximity sensors (sort of like a 1 pixel Microsoft Kinect). These proximity sensors enable the Surface to sense users as they move around the tabletop, and detect a user’s hands and arms above the display area of the Surface. Not only are these sensors inexpensive and simple to configure, but also they enable an integrated hardware solution, without requiring any markers, cameras, or other sensing devices external to the display platform itself. 

As Medusa has an awareness of users’ locations, it can for example, identify touch points by user, and disambiguate between touches made with left or right hands. It can also make use of the touch-based information provided from the Surface to map touch points to specific users (as well as identify which hand they used, right or left), even in multi-user scenarios.

Using all of this information, there are an infinite number of ways that multi-touch interaction with a horizontal display can be enhanced and augmented. In the video below, I (along with Tovi Grossman), demonstrate a few of the techniques that we explored.

Abstract:

We present Medusa, a proximity-aware multi-touch tabletop. Medusa uses 138 inexpensive proximity sensors to: detect a user’s presence and location, determine body and arm locations, distinguish between the right and left arms, and map touch point to specific users and specific hands. Our tracking algorithms and hardware designs are described. Exploring this unique design, we develop and report on a collection of interactions enabled by Medusa in support of multi-user collaborative design, specifically within the context of Proxi-Sketch, a multi-user UI prototyping tool. We discuss design issues, system implementation, limitations, and generalizable concepts throughout the paper.