Plenoptic SamplingLight Field MethodsBackgroundTwo Plane ParametrizationReconstructionSlide 6FormulationA Key ConceptContributionConvolutionSpectral SupportRadianceSlide 13Research QuestionScene with constant depthSlide 16Spatially varying depth modelScene ImagesReconstruction at constant depthSample RenderingsMinimum Sampling RateJoint Image and Geometry SpaceSlide 23Results (more in the paper)Plenoptic SamplingJian Huang, CS 594, Spring 2002Light Field Methods•For image-based rendering algorithms based on plenoptic function, some way of discrete sampling is always used•What is the optimal sampling? Can Nyquist sampling theory be applied?•Plenoptic Sampling, Proc. SIGGRAPH’2000, Jin-Xiang Chai, Xin Tong, Shing-Chow Chan, Heung-Yeung Shum, Microsoft Research, China–Minimal sampling rate for light fieldsBackground•Texture mapping, very accurate geometry and only a few images•Image-based methods with depths: –3D warping, LDI (layered depth image), view morph, interpolation–A few images with depth information•Light field methods–Do not assume any depth information: light fields, lumigraph, concentric mosaics–Rely on over-sampling (large amount images)–Intensive data acquisition, storage and processingTwo Plane ParametrizationObjectFocal plane (st)Camera plane (uv)Reconstruction•Plenoptic Sampling•How many samples of the plenoptic function (e.g., from a 4D light field) and how much geometrical and textural information are needed to generate a continuous representation of the plenoptic function?•Two specific goals:–Minimum sampling rate for light field rendering–Minimum sampling curve in joint image and geometry spaceFormulation•A high-dimension signal processing problem•Assumption–Lambertian surface–Uniform sampling geometry or lattice•Study:–Spectral support of light field signals–Not closed-form spectral representationA Key Concept•The spectral support of a light field signal is bounded by only the minimum and maximum depths, irrespective of how complicated the spectral support might be because of depth variations in the scene. •Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering.Contribution•The minimum sampling rate of light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval without any overlap.•Using more depth information, plenoptic sampling in the joint image and geometry space allows us to greatly reduce the number of images needed. •The relationship between the number of images and the geometrical information under a given rendering resolution can be described by a minimum sampling curve.•This minimal sampling curve serves as the design principles for IBR systems, bridging the gap between image-based rendering and traditional geometry-based rendering.Convolution•Sampling a continuous light field function, l, with sampling pattern, p, reconstruction filter (kernel), r, reproducing images i.Spectral Support•Let z(u,v,s,t) to be the depth function, i.e. geometry•The same point is views in camera 0 and t as point v and v’.•Assuming Lambertian surface, each epi-polar line is of uniform colorRadiance•The radiance received at camera location (s,t) is:•With the Fourier transform being: (complicated to compute)Spectral Support•Using a rectangular sampling lattice, the sampled function:•L has to be bandlimited.•Need to use a sampling rate higher than Nyquist for alias-free sampling.Research Question•Is there an optimal reconstruction function (like sinc in conventional signal processing)?•The design of such a function is related to the depth function. How exact should depth be captured and recovered?Scene with constant depth•Choosing the first frame as reference, l(u,v,0,0).Spectral Support•Focusing on sampling in (v,t) sub-spaceSpatially varying depth model•Straightforward to observe that the spectral support of a scene of depth range [zmin, zmax] is bounded by two lines:Scene ImagesReconstruction at constant depth•Assuming a constant depth in reconstruction•Optimal depth:Sample Renderings•At min depth, optimal depth, average depth and max depth:Minimum Sampling Rate •How tight can you pack:Joint Image and Geometry Space•With a precise geometry, can decompose a scene into multiple layers of depthsJoint Image and Geometry Space•With depth uncertainty, for instance, a noisy depth imageObviously, the curve will be scene dependent!Results (more in the
View Full Document
Unlocking...