Dark Flash PhotographyDilip Krishnan∗Rob FergusDept. of Computer Science, Courant Institute, New York UniversityF A R LFigure 1: Our camera and flash system offers dazzle-free photography by hiding the flash in the non-visible spectrum. A pair of imagesare captured at a blur-free shutter speed, one using a multi-spectral flash (F), the other using ambient illumination (A) which in this caseis 1/100th of that required for a correct exposure. The pair are combined to give an output image (R) which is of comparable quality to areference long exposure shot (L). The figures in this paper are best viewed on screen, rather than in print.AbstractCamera flashes produce intrusive bursts of light that disturb or daz-zle. We present a prototype camera and flash that uses infra-red andultra-violet light mostly outside the visible range to capture picturesin low-light conditions. This “dark” flash is at least two ordersof magnitude dimmer than conventional flashes for a comparableexposure. Building on ideas from flash/no-flash photography, wecapture a pair of images, one using the dark flash, other using thedim ambient illumination alone. We then exploit the correlationsbetween images recorded at different wavelengths to denoise theambient image and restore fine details to give a high quality result,even in very weak illumination. The processing techniques can alsobe used to denoise images captured with conventional cameras.Keywords: Computational Photography, Dark Flash, Multi-Spectral Imaging, Spectral Image Correlations∗Email: [email protected] IntroductionThe introduction of digital camera sensors has transformed pho-tography, permitting new levels of control and flexibility over theimaging process. Coupled with cheap computation, this has precip-itated a wide range of novel photographic techniques, collectivelyknown as Computational Photography. Modern camera sensors, bethey in a cellphone or a high-end DSLR, use either a CCD or CMOSsensor based on silicon. The raw sensor material responds to lightover a wide range of wavelengths, typically 350–1200nm. Coloreddyes are deposited onto the sensor pixels in a Bayer pattern, result-ing in 3 groups of pixels (red, green and blue). Each responds toa limited range of wavelengths, approximating the sensitivities ofthe three types of cone cell in our retina. However, silicon is highlysensitive to infra-red (IR) wavelengths and it is difficult to manu-facture dyes that have sufficient attenuation in this region, thus anextra filter is placed on top of most sensors to block IR light. Thisgives a sensor that records only over the range 400-700nm, match-ing our own color perception, but a considerable restriction of theintrinsic range of the device.One solution to capturing photographs in low light conditions is touse a flash unit to add light to the scene. Although it provides thelight to capture otherwise unrecordable scenes, the flash makes thephotographic process intrusive. The sudden burst of light not onlyalters the illumination but disturbs any people present, making themaware that a photo has just been taken and possibly dazzling them ifthey happen to be looking toward the camera. For example, a groupphoto in a dark restaurant or bar using a bright camera flash leavesthe subjects unable to see clearly for some moments afterward.In this paper we introduce a camera/flash system that is basedaround off-the-shelf consumer equipment, with a number of mi-nor modifications. First, the camera is a standard DSLR with theIR-block filter removed, thus restoring much of the original spectralrange of the sensor. Second, we use a modified flash that emits lightover a wider spectral range than normal, which we fi lter to removevisible wavelengths. This dark flash allows us to add light to thescene in such a way that it can be recorded by the camera, but notby our own visual system. Using the dark flash we can illuminatea dimly lit scene without dazzling people present, or significantlydisturbing those around. Furthermore, it allows a fast shutter speedto be used, thus avoiding camera shake. However, the difficulty isthat people want images with colors that match their visual expe-rience and this will not be the case for images captured using thedark flash.To overcome this, we acquire a pair of images in the man-ner of flash/no-flash photography [Eisemann and Durand 2004;Petschnigg et al. 2004], one using the dark flash and the second us-ing ambient illumination alone. For the latter to be blur-free a fastshutter speed must be used, resulting in high noise levels in dimlight. A key observation is that if the non-visible and visible chan-nels are close in wavelength, strong correlations will exist betweenthem. We introduce a novel type of constraint that exploits the cor-relations between spectral bands. Using this constraint, the edgestructure of the dark flash image can be used to remove the noisefrom the ambient image, yielding a high quality result that lacks theshadow and specularity artifacts present in the flash image.We also show how our camera/flash hardware and spectral con-straints can be used in a range of additional applications, including:inferring spectral reflectance functions of materials in the scene anddenoising individual color channels of images captured with stan-dard cameras.1.1 Related workOur approach can be regarded as a multi-spectral version ofthe flash/no-flash technique introduced by [Agrawal et al. 2005],[Petschnigg et al. 2004] and [Eisemann and Durand 2004].[Agrawal et al. 2005] focused on the removal of flash artifacts butdid not apply their method to ambient images containing signifi-cant noise, unlike [Petschnigg et al. 2004] and [Eisemann and Du-rand 2004]. The two latter approaches are similar in that they use across-bilateral (also known as joint-bilateral) filter and detail trans-fer. However, [Petschnigg et al. 2004] attempt to denoise the am-bient, adding detail from the flash, while [Eisemann and Durand2004] alter the flash image using ambient tones.The closest work to ours is that of [Bennett et al. 2007], who showhow video captured in low-light conditions can be denoised usingcontinuous IR illumination. However, they make use of temporalsmoothing to achieve high quality results, something not possiblein our photography setting. [Wang et al. 2008a] show how IR illu-mination can be used to relight faces in well-lit scenes. Both theseworks differ from ours in a number of ways: (i) they use complexoptical