LowLow--Cost MultiCost Multi--Touch Sensing through Touch Sensing through Frustrated Total Internal ReflectionFrustrated Total Internal ReflectionJefferson Y. HanMedia Research LaboratoryNew York UniversityReviewer: Paul VarcholikUniversity of Central FloridaSingleSingle--Touch InterfacesTouch Interfacesy One finger at a timey One user at a timey Examples:◦ ATM◦ Point-of-Sale◦ Restaurant Ordering SystemsMultiMulti--Touch InterfacesTouch Interfacesy Multiple simultaneous points of contacty Inherently Multi-User◦ Interactive Walls◦ Tabletopsy Just coming to the market◦ Perspective Pixel◦ Microsoft SurfaceMulti-Touch VideoMulti-Touch Interaction Experiments2006 Jefferson Y. HanFrustrated Total Internal ReflectionFrustrated Total Internal Reflectiony When light encounters an interface to a medium with a lower index of refraction (e.g. glass to air), the light becomes refracted.y The extent of refraction depends on the angle of incidence, and beyond a certain critical angle, it undergoes total internal reflection (TIR).y Fiber Optic cable is a common technology that employs TIR.y However, another material at the interface can frustrate this TIR, causing the light to escape the waveguide there instead.Frustrated Total Internal ReflectionFrustrated Total Internal ReflectionImplementation Implementation ––Primary Primary ComponentsComponentsy Acrylic Sheet (1/4” or thicker)y Infra-Red LEDsy IR-Detecting Camera (modified webcam)y Projectory Diffuser (projection surface overlay)y Frame with Light Bafflesy ComputerWhy this Approach?Why this Approach?y Inexpensive◦ My prototype is at $542 (not including the $2,300 projector, computer, or miscellaneous tools and “workbench” materials).y Scalable Size◦ From handheld to full wall displays.y Relatively simple to constructDownsides to this ApproachDownsides to this Approachy Space Behind Projection Surface Required◦ Rear-Projection◦ Cameray Can Be Mitigated By:◦ Short-Throw Projector◦ Multiple Camerasy Doesn’t tell you what finger (or what user) produced the contact.y Doesn’t provide “hover” information (might not be a downside).Practical Concerns (1)Practical Concerns (1)y Placing the diffuser behind the acrylic causes disparity between the display and interaction surfaces.y Mitigation:◦ Make the diffuser the interaction surface by placing it on top of the acrylic (has consequences).Practical Concerns (2)Practical Concerns (2)y Oils and Sweat will contaminate the acrylic.y Mitigation:◦ Make the diffuser the interaction surface by placing it on top of the acrylic (has consequences).Practical Concerns (3)Practical Concerns (3)y Dry skin, directly against acrylic, generates weaker blobs.y Why?◦ Tiny air gaps exist between your finger and the surface. Moisture provides better coupling.y Mitigation:◦ Press Harder◦ Moisten Skin◦ Use a compliant surface (e.g. silicone) between the acrylic and the diffuser.Practical Concerns (4)Practical Concerns (4)y Modern projectors have a 2:1 throw ratio (2 feet of distance for every 1foot of diagonal display)y Mitigation:◦ Fold the image using mirrors (I couldn’t make this work well).◦ Use a short throw projector (expensive).Practical Concerns (5)Practical Concerns (5)y Modern projectors typically display horizontally, and have cooling systems designed for this orientation.y Mitigation:◦ Use mirrors to angle the image upward –tabletop displays – not applicable for wall displays).Practical Concerns (6)Practical Concerns (6)y Need baffles on the front of the surface to reduce “backlighting” of your hand or body as it draws near the surface. This is caused from IR light “leaking” from the edges of the acrylic.y Baffles on the underside of the acrylic are necessary to eliminate “hot spots”.Practical Concerns (7)Practical Concerns (7)y Mounting camera and projector –especially a concern for portability and quickly recalibrating the setup.Practical Concerns (8)Practical Concerns (8)y Software – Free/Open Source software has a (not completely unjustified) reputation of being poorly constructed.y TouchLib – I think this is a decent start –but is very rough around the edges.Practical Concerns (9)Practical Concerns (9)y The least expensive camera option (webcam – which is also the most easily accessed via software) filters IR light and allows visible light. We want to reverse this – allow IR light and filter visible light.y The webcam has to be opened, and the IR filter removed. Then the webcam has to be fitted with a visible light filter.y Problems:◦ Hard (or impossible) to open some webcams.◦ Some webcam IR filters are painted on.◦ Materials for visible light filter:x Exposed film negativex Floppy diskx Trash bag (multiple ply)x Bona-fide, commercial IR-pass filterPrototypePrototypePrototypePrototypePrototype Prototype ––Parts ListParts Listy 24”x36”x3/8” Acrylic Sheety Drafting Tabley 30”x42”x3/4” MDF Sheety 32 Osram SFH485 880nm IR LEDs – 4 sets wired in parallel of 8 LEDs wired in seriesy 12v AC/DC power supplyy Microsoft LifeCam VX-6000 (71degree wide-angle lens, 1.3MP max resolution, up to 30fps,USB 2.0)y Rosco Gray Rear-Projection Screeny 3M DMS-700 Short-Throw Projectory Silicone Rubber (Sort-A Clear 40 )y Misc. Cables/Resistors/Switches/Hardwarey MirrorOther ApplicationsOther Applicationsy Tangible Interfaces◦ Cell Phones◦ Credit Cards◦ MP3 Players◦ Anything with a fiducial/bar code on the bottom or ity Biometric Sensing (e.g. finger/palm prints)y Requires Visible-Light