MIT Media Lab Computational Camera Photography Camera Culture Ramesh Raskar MIT Media Lab http CameraCulture info Plan for Today Computational Illumination Introduction to Lightfields Assignment 2 Optics A Virtual Optical Bench B Lightfields Check Wiki for Reading Material Pl edit add URLs figures comments etc Everyone on comp cam mailing list Listeners Pl send a topic you would like to present 15 20 mins Youtube videos on camera tutorial DoF etc http www youtube com user MPTutor Final Project Ideas User interaction device Camera based Illumination based Photodetector or line scan camera Capture the invisible Tomography for internals Structured light for 3D scanning Fluorescence for transparent materials Cameras in different EM other spectrum Wifi audio magnetic haptic capacitive Visible Thermal IR segmentation Thermal IR emotion detection motion detector Multispectral camera discriminating camelsand Illumination Multi flash with lighfield Schielren photography Strobing and Colored strobing External non imaging sensor Camera with gyro movement sensors find identity of user Cameras with GPS and online geo tagged photo collections Interaction between two cameras with lasers on board Optics Lightfield Coded aperture Bio inspired vision Time Time lapse photos Motion blur Sample Final Projects Schlieren Photography Best project award Prize in 2008 Camera array for Particle Image Velocimetry BiDirectional Screen Looking Around a Corner theory Tomography machine Auto Focus Contrast method compares contrast of images at three depths if in focus image will have high contrast else not Phase methods compares two parts of lens at the sensor plane if in focus entire exit pupil sees a uniform color else not assumes object has diffuse BRDF Homeworks Submit to class website Commented Source code Input images and output images PLUS intermediate results CREATE a webpage and send me a link Ok to use online software HDRshop Update results on Flickr group page Second Homework Option A Extending Andrew Adam s Virtual Optical Bench a Create new optical bench Ability to add new elements b Form images Integrate light from multiple rays Show image effects DoF LF etc Courtesy of Andrew Adams Used with permission Synthetic aperture videography Vaish V et al Using Plane Parallax for Calibrating Dense Camera Arrays Proceedings of CVPR 2004 Courtesy of IEEE Used with permission 2004 IEEE Second HW b Lightfield Photography Translate camera and take photos Show refocussing and see thru effects http lightfield stanford edu lfs html Part 1 Create images with plane of focus at different depth Create images with variable depth of field just use fewer images Create see thru effects just small depth of field Find depth using max contrast operator Part 2 Images with SLANTED plane of focus See thru effect by elminating foreground color pixels Extra credit Create new bokeh point spread function Use high depth complexity colorful point specular sphere objects Second HW c Lightfield Photography Do ALL in software using Virtual Optical Bench Translate camera and take photos Show refocussing and see thru effects http lightfield stanford edu lfs html Part 1 Create images with plane of focus at different depth Create images with variable depth of field just use fewer images Create see thru effects just small depth of field Find depth using max contrast operator Part 2 Images with SLANTED plane of focus See thru effect by elminating foreground color pixels Extra credit Create new bokeh point spread function Use high depth complexity colorful point specular sphere objects Computational Illumination Part 2 What are annoyances in photography Why CCD camera behaves retroreflective Youtube videos on camera tutorial DoF etc http www youtube com user MPTutor Poll When Google Earth Live Why Commercially exploited pervasive recording http www youtube com watch v J7qE6frzz8 http www newscientist com article dn17854 live videomakes google earth cities bustle html Measuring Light I say that if the front of a building or any open piazza or field which is illuminated by the sun has a dwelling opposite to it and if in the front which does not face that sun you make a small round hole all the illuminated objects will project their images through that hole and be visible inside the dwelling on the opposite wall which may be made white and there in fact they will be upside down and if you make similar openings in several places in the same wall you will have the same result from each Hence the images of the illuminated objects are all everywhere on this wall and all in each minutest part of it The notebooks of Leonardo da Vinci Next several slides by Ashok Veeraraghavan What do we see 3D world 2D image Point of observation Images courtesy of Stephen E Palmer Used with permission Figures Stephen E Palmer 2002 What do we see 3D world 2D image Painted backdrop Images courtesy of Stephen E Palmer Used with permission The Plenoptic Function Figure removed due to copyright restrictions Q What is the set of all things that we can ever see A The Plenoptic Function Adelson Bergen Let s start with a stationary person and try to parameterize everything that he can see Grayscale snapshot Figure removed due to copyright restrictions P is intensity of light Seen from a single view point At a single time Averaged over the wavelengths of the visible spectrum can also do P x y but spherical coordinate are nicer Color snapshot Figure removed due to copyright restrictions P is intensity of light Seen from a single view point At a single time As a function of wavelength A movie Figure removed due to copyright restrictions P t is intensity of light Seen from a single view point Over time As a function of wavelength Holographic movie Figure removed due to copyright restrictions P t VX VY VZ is intensity of light Seen from ANY viewpoint Over time As a function of wavelength The Plenoptic Function Figure removed due to copyright restrictions P t VX VY VZ Can reconstruct every possible view at every moment from every position at every wavelength Contains every photograph every movie everything that anyone has ever seen Sampling Plenoptic Function top view Ray Let s not worry about time and color P VX VY VZ 5D 3D position 2D direction Courtesy of Rick Szeliski and Michael Cohen Used with permission Slide by Rick Szeliski and Michael Cohen Ray No Occluding Objects 4D P VX VY VZ 2D position 2D direction The space of all lines in 3 D space is 4D Courtesy of Rick Szeliski