MAS 963 Computational Camera and Photography Fall 2009 Class 6 Cameras for HCI Prof Ramesh Raskar Scribe Anonymous MIT student October 16 2009 Class 6 Cameras for HCI Exam Light Fields continued A light eld can be represented in 4 dimensions The sensor with x and y and the Lens with and Light elds are a complete representation of the the light rays captured by the lens Therefore parameters like focus zoom and aperture size can be changed after the photo is taken Light Field with Pinhole Mask Subject Lens Pinhole Mask Sensor Figure 1 Light Field Camera with Pinhole Mask not to scale Figure 1 shows an example of a light eld camera with a pinhole mask in front of the sensor If we look at the sensor in 1D and assume an total x resolution of 900 pixels and a resolution of 9 the local x resolution is 100 The disadvantages are therefore a loss in resolution and a loss of light as most of the light gets blocked by the mask Therefore while this model is very clean on a theoretical level it is ine cient to apply in the real world How does the spacing of the mask from the sensor e ect the captured light eld regions Figure 2 depicts the e ects of the spacing between the sensor and the mask When the mask is correctly spaced the blobs on the sensor barely touch each other When it is spaced too far away artifacts appear due to overlapping blobs If the mask is too close empty space between the blobs results in wasted pixels 2 1 Pinhole Mask a Sensor Pinhole Mask Sensor b Pinhole Mask Sensor c Figure 2 E ect of distance between mask and sensor a correct spacing between sensor and mask b mask is too close sensor pixels are wasted c mask is too far away blobs overlap f number The f number is the ratio of the focal length over the lens diameter A larger lens diameter results in a smaller f number Example If the lens has a diameter of 25 mm and the focal length is 50 mm the F number is 2 If the lens diameter is 12 5 mm the f number is 4 When the f number is decreasing by factor 2 the area is decreasing by factor 4 therefore only 1 4 of the light reaches the sensor Going down by one f stop means going down from 2 to 2 8 after that 4 2 3 square 2 and 5 6 4 square 2 same angle for same f number cone of light for each pixel We want a ratio of focal length to lens diameter which is equal equal to the mask distance to blob size If that is matched the blobs will barely touch each other if not there will be overlap or lost pixels see 2 Pinhole Photography Problems Because of the pinhole very little light reaches the sensor therefore long exposure Also the image is blurred because of di raction single point in the world maps to a blurred spot on the sensor Analogy to water hose when the size of the opening in the water hose becomes comparable to the size of the water molecule it will start to spray Wavelength of light 400 to 700 nm visible light Green light is 500 nm 5 microme ters If we assume a 1mm pinhole 1000 micrometers below 500 micrometers becomes too small di raction artifacts This problem is also common with cellphone camera lenses A 2 mm aperture results in the light spreading out about 20 nm If the sensor pixel size is 5 microns the blur is larger than the pixel size A light eld camera with a pinhole mask has the same problems as a traditional pinhole camera 0 1 Light Field with Lenslet Array When using a lenslet array the setup is similar to a pinhole camera with an array of lenses instead of pinholes see Figure 3 There are two special constraints we have to consider Because it is a light eld camera we want to create a lens which forms an image of a lens on the sensor This results in very tight tolerances for the distance of the lenslet array to 2 2 Subject Lens Lenslet Array Sensor Figure 3 Light Field Camera with lenslet array not to scale the sensor as a wrong distance will result in a blurred image on the sensor Due to this constraint building such a camera is very challenging 10 Another constraint is matching the main lenses f number to the lenslet arrays f numbers to get an optimal directional resolution If the f number is too low the resulting images formed by the lenslet array will overlap if it is too high pixels on the sensor will be wasted 0 2 Light eld with aperture mask Figure 4 A lens is equivalent to a set of pinholes each with its own prism bending the light and forcing it to converge at one point behind the camera Image courtesy by Ahmed Kirmani A lens is can be modeled as an array of pinholes with corresponding prisms see Figure 4 So if we take a light eld image with a camera array it s the same setup without the prisms By shifting the image mathematically we create the set of corresponding prisms Considering this we can obtain a light eld by taking multiple photos with parts of the 2 3 lens blocked 3 0 3 Light eld with mask placed between sensor and lens When taking a 2d fourier transform of an image most of the energy is in the low frequencies center If we replace the pinhole array in front of the sensor with a high frequency mask a lot of the energy is in the high frequencies similar to heterodyning radio signals by mixing a signal with a high frequency carrier If we then rearrange the individual tiles into 4D planes and compute the inverse Fourier transform we get the light eld The bene t over a pinhole array is more light hitting the sensor almost 50 of the light goes through therefore less di raction artifacts Output is almost the same as a lenslet array after computation Aliasing happens in the frequency domain unless pre ltering 0 4 Hadamard multiplexing Example several bags with di erent weights To determine the weight of the di erent bags either weigh every one and therefore get each weight Also possible weigh multiple bags at once useful if scale works well in the mid range not too low or too high weight Put a group of them 3 of them simple linear system w1 w2 w2 w3 w3 w1 take about half of the bags take measurement other half take measurement Total of 9 measurements with 4 or 5 bags put together Disadvantage lot of computation Hadamard and cosine is one and the same thing Mask can be optimized in many ways between all 0 blocking and all 1 transparent Talk by Matt Hirsch about BiDi screen BiDi screen was inspired by lcd screens with embedded optical sensing …