IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 1Coded Strobing Photography: Compressive Sensingof High Speed Periodic VideosAshok Veeraraghavan∗, Member, IEEE, Dikpal Reddy∗, Student Member, IEEE,and Ramesh Raskar, Member, IEEEAbstract—We show that, via temporal modulation, one canobserve and capture a high-speed periodic video well beyond theabilities of a low-frame-rate camera. By strobing the exposurewith unique sequences within the integration time of each frame,we take coded projections of dynamic events. From a sequenceof such frames, we reconstruct a high-speed video of the high-frequency periodic process. Strobing is used in entertainment,medical imaging, and industrial inspection to generate lowerbeat frequencies. But this is limited to scenes with a detectablesingle dominant frequency and requires high-intensity lighting.In this paper, we address the problem of sub-Nyquist samplingof periodic signals and show designs to capture and reconstructsuch signals. The key result is that for such signals, the Nyquistrate constraint can be imposed on the strobe rate rather thanthe sensor rate. The technique is based on intentional aliasingof the frequency components of the periodic signal while thereconstruction algorithm exploits recent advances in sparse rep-resentations and compressive sensing. We exploit the sparsity ofperiodic signals in the Fourier domain to develop reconstructionalgorithms that are inspired by compressive sensing.Index Terms—Computational imaging, high speed imaging, com-pressive sensing, compressive video sensing, stroboscopy.I.INTRODUCTIONPeriodic signals are all around us. Several human and animalbiological processes such as heart-beat, breathing, several cel-lular processes, industrial automation processes and everydayobjects such as hand-mixer and blender all generate periodicprocesses. Nevertheless, we are mostly unaware of the innerworkings of some of these high-speed processes because theyoccur at a far greater speed than can be perceived by the humaneye. Here, we show a simple but effective technique that canturn an off-the-shelf video camera into a powerful high-speedvideo camera for observing periodic events.Strobing is often used in entertainment, medical imaging andindustrial applications to visualize and capture high-speedvisual phenomena. Active strobing involves illuminating thescene with a rapid sequence of flashes within a frame time. Theclassic example is Edgerton’s Rapatron to capture a golf swing[13]. In modern sensors, it is achieved passively by multiple-exposures within a frame time [36][28] or fluttering [29]. Weuse the term ‘strobing’ to indicate both active illumination andpassive sensor methods.In case of periodic phenomenon, strobing is commonly usedto achieve aliasing and generate lower beat frequencies. Whilestrobing performs effectively when the scene consists of asingle frequency with a narrow sideband, it is difficult to visu-alize multiple or a wider band of frequencies simultaneously.∗Ashok Veeraraghavan and Dikpal Reddy contributed equally to this work.Periodic phenomenon with unknown period P (say 16 ms)Video camera w/ frame rate fs= 25fpsEvery frame is modulated U = 80 times with a unique binary code by opening & closing the shutterCapture M = 125 framesin 5sComputationally reconstruct N = 10000 framesCoded Strobing SchematicStructured sparse recoveryTimeTimeCoded Strobing: Time DomainCoded Strobing: Frequency DomaintP = 16msAt each pixel, the periodic signal is temporally modulated with a binary codeTFrame= Frame Duration = 40msAt a pixel, the M observed intensity values are linear combinations of the periodic signal’s sparse Fourier coefficients Binary code of length U=80Measure Linear Combinations4fPStructured Sparsity Enforcing Reconstruction Algorithm0fMax- fMaxfP=1/P2fP-fP-2fP4fPfFig. 1: CSC: A fast periodic visual phenomenon is recorded bya normal video camera (25 fps) by randomly opening and closingthe shutter at high speed (2000 Hz). The phenomenon is accuratelyreconstructed from the captured frames at the high-speed shutter rate(2000 fps).Instead of direct observation of beat frequencies, we exploit acomputational camera approach based on different samplingsequences. The key idea is to measure appropriate linearcombinations of the periodic signal and then decode the signalby exploiting the sparsity of the signal in Fourier domain.We observe that by coding during the exposure duration ofa low-frame-rate (e.g., 25 fps) video camera, we can takeappropriate projections of the signal needed to reconstruct ahigh-frame-rate (e.g., 2000 fps) video. During each frame, westrobe and capture a coded projection of the dynamic eventand store the integrated frame. After capturing several frames,we computationally recover the signal independently at eachpixel by exploiting the Fourier sparsity of periodic signals.Our method of coded exposure for sampling periodic signalsis termed ‘coded strobing’ and we call our camera the ‘codedstrobing camera’ (CSC). Figure 1 illustrates the functioningof CSC.A. Contributions• We show that sub-Nyquist sampling of periodic visualsignals is possible and that such signals can be capturedand recovered using a coded strobing computationalcamera.• We develop a sparsity-exploiting reconstruction algorithmand expose connections to Compressive Sensing.• We show that the primary benefit of our approach overtraditional strobing is, increased light-throughput andIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2ability to tackle multiple frequencies simultaneously post-capture.B. Benefits and limitationsThe main constraint for recording a high-speed event islight throughput. We overcome this constraint for periodicsignals via sufficient exposure duration (in each frame) andextended observation window (multiple frames). For well-lit non-periodic events, high-speed cameras are ideal. Fora static snapshot, a short exposure photo (or single frameof the high-speed camera) is sufficient. In both cases, lightthroughput is limited but unavoidable. Periodic signals canalso be captured with high-speed camera. But one will needa well-lit scene or must illuminate it with unrealistic brightlights. For example, if we use a 2000 fps camera for vocalcord analysis instead of strobing using a laryngoscope, wewould need a significantly brighter illumination source andthis creates the risk of burn injuries to the throat. A saferoption would be 25 fps