A Multi-Layered Display with Water DropsPeter C. Barnum Srinivasa G. NarasimhanRobotics Institute, Carnegie Mellon UniversityTakeo Kanade∗(a) Video (b) Text (c) Games (d) ImagesFigure 1: By using precisely controlled valves and a projector-camera system, we create a vibrant, multi-layered water drop display.The display can show static or dynamically generated images on each layer, such as text, videos, or even interactive games.AbstractWe present a multi-layered display that uses water drops as vox-els. Water drops refract most incident light, making them excellentwide-angle lenses. Each 2D layer of our display can exhibit arbi-trary visual content, creating a layered-depth (2.5D) display. Oursystem consists of a single projector-camera system and a set oflinear drop generator manifolds that are tightly synchronized andcontrolled using a computer. Following the principles of fluid me-chanics, we are able to accurately generate and control drops sothat, at any time instant, no two drops occupy the same projectorpixel’s line-of-sight. This drop control is combined with an algo-rithm for space-time division of projector light rays. Our prototypesystem has up to four layers, with each layer consisting of an rowof 50 drops that can be generated at up to 60 Hz. The effective reso-lution of the display is 50× projector vertical-resolution × numberof layers. We show how this water drop display can be used fortext, videos, and interactive games.CR Categories: I.3.1 [Computer Graphics]: Hardwarearchitecture—Three-dimensional displays; B.4.2 [Input/Output anddata communications]: Input/Output devices—Image display1 IntroductionWater drops are transparent and serve as tiny fish-eye lenses. Theselenses collect light and refract it out across a wide range of anglesand can be among the brightest elements in an environment. Bytaking advantage of water drops’ tendency to refract light and byilluminating collections of drops, vibrant displays can be created.Water drop displays have been used to show anything from abstractpatterns to movie-theater quality video. These displays have either∗e-mail: {pbarnum,srinivas,tk}@cs.cmu.edubeen high-resolution 2D [Araki et al. 1991; Palovuori and Rakko-lainen 2004], or low resolution 3D [Eitoku et al. 2009]. In thiswork, we have created a high-resolution, multi-layered (2.5D) dropdisplay that can show images, videos, or games.To create the display, we synchronized a projector, a camera, andmultiple linear drop-emitter manifolds. The first step is to accu-rately generate layers of drops. Each layer is created by a linearmanifold of emitters, controlled by a single solenoid valve. Whenthe valve is actuated, the manifold emits a row of water drops thatfreely fall down a vertical plane. The timings of the drop emissionsfrom different manifolds are offset, so that at any instant in time, notwo drops occlude each other from the point of view of a projector.The second step is to find the drops’ positions and illuminate themwith the correct colors. The camera captures several images of thelocations of the falling drops. Based on the detected drop locations,the projector quickly switches images, so that drops from each layerare illuminated with the correct colors. Figure 1 shows exampleusages. The display can show static images like the text in Figure1 (b) or the solid-color images in Figure 1 (d). It can also play pre-recorded videos, like the video of the woman in Figure 1 (a), oreven interactive content, as with the Tetris variant in Figure 1 (c).Previous 2D and sparse 3D displays offer an enjoyable experi-ence, but they do not offer functionality beyond a touchscreen ora laser-light show. But a high-resolution 3D drop display wouldbe uniquely useful. Since water is not solid, both the outside andthe inside of the display volume could be directly touched, openingup new interaction possibilities. By stacking more layers, our 2.5Ddisplay could scale to become a high-resolution 3D display.2 Previous WorkWater drop displays can be compared by asking two questions.First, how accurately can the drops’ locations be predicted or con-trolled? A flat 2D display requires no control, but a 3D displaywith drops occluding each other at different times needs preciselypositioned drops. Second, how quickly and accurately can the lightsource be controlled? The faster and more accurate the light source,the more tolerance the display has to drops occluding each other.The simplest drop displays control only the light source. Fog screendisplays [Araki et al. 1991; Palovuori and Rakkolainen 2004; Leeet al. 2007] use a sheet of microscopic droplets and a standardprojector to create high-resolution, walk-through projector screens.But because they have no control over individual drops, they requirea different projector for each screen.In addition to light-control, some displays selectively turn on andoff fountains of water, such as Disney’s Fantasmic show [Disney1992-2010] and the Submerging Technologies [Dietz et al. 2006]pieces. Such limited water control allows for interesting interac-tions and large-scale performances. The purpose of these displaysis to create specific artistic effects, so they should not be evaluatedonly with quantitative metrics. However, this limited water controldoes not allow complexity much beyond that of fog screens.Instead of controlling the light source, some displays, such as[Pevnick 1981], use a constant light source and arrays of individu-ally controlled drop emitters to print out falling images. This hasthe advantage of spatial scalability. But as the drops are constantlyfalling, individual images can only be shown temporarily, and thisdisplay cannot be used for video.As shown by the previous examples, fun and colorful water displayscan be made by controlling only the water or only the illumination.But the most versatile types use a combination. Some such displaysuse uncontrolled drops, but detect the drops’ positions in real time.Section 3 of [Barnum et al. 2009] shows a method of detecting,tracking, and illuminating individual water drops. But due to theslow reaction time of the camera-projector system, it is limited toonly a few drops at a time, and thus cannot create coherent images.But as suggested in the patent [Perlin and Han 2006], a high-speedadaptively-controlled laser could show arbitrary 3D shapes in par-ticulate media (such as dust). It will be interesting to see if and howthis can be implemented.Instead of finding drops