11Image-Based RenderingThomas FunkhouserPrinceton UniversityC0S 426, Fall 20012Traditional ApproachesTraditionalComputerVisionTraditionalComputerGraphics(McMillan)3Image-Based Rendering (IBR)ComputerVisionComputerGraphicsImage-Based RenderingAnalyze &ReprojectSimulate(McMillan)4IBR Rendering Pipeline(McMillan)5Image-Based Representations• Plenoptic function (7D):o Describes the radiance traveling along a ray- to/from any point (x, y, z),-inanydirection(φ, θ),- at any frequency (λ),-atanytime(t)F(x, y, z, φ, θ, λ, t)(x,y,z)(φ,θ)6Image-Based RepresentationsF(x, y, z, φ, θ, λ, t)7D6D5D4D3D2DIdealConsider only 3 frequencies (RGB)Consider only one time instant (static scene)Consider only viewpoints inside/outside sceneConsider one dimension fewer directions/positionsConsider viewpoints at finite set points or angles27View InterpolationDerived FrameReference FrameReference Frame• Create novel images by resampling photographso Reference images sample 5D plenoptic function8View Interpolation• Method:o Warp nearby reference images to novel viewpointo Blend warped imagesThis is just a morphwhere the warp is defined bypixel correspondences!R2R1N9View Interpolation• How find pixel correspondences:o Disparityo Sparse featureso Depth at every pixelo Coarse 3D model(Szeliski)LeftRightDisparity10View Interpolation• Problem:o Changes in visibilityo Disocclusions(McMillan)11Disocclusions• Partial solutions:o Use more photographso Fill holes by interpolating nearby pixels(McMillan)12Disocclusions• Better solutions (when possible):o Multiple samples per pixel at different depths313Disocclusions• Better solutions (when possible):o Multiple samples per pixel at different depths(Popescu)Reference Image Warped Depth Image14Disocclusions• Better solutions (when possible):o Multiple samples per pixel at different depths(Popescu)Warped Layered Depth ImageReference Image15Light Field / Lumigraph• If observer stays in free space,plenoptic function reduces to 4Do Exterior of the convex hull of an objecto Interior of an environment(Levoy96)F(r, α, φ, θ)16Representing a Light Field• Two-plane parameterization (4D)(Levoy96)17Representing a Light Field(Levoy96)18Two Interpretations of a Light Field(Levoy96)419Creating a Light Field(Levoy96)20Capturing a Light Field(Stanford University)21Capturing a Light Field(Bennett Wilburn, Michal Smulski, Mark Horowitz)22Rendering a Light Field• Resampling problemo Interpolationo Avoid aliasing(Gortler96)23Rendering a Light Field• Resampling with approximate geometryo Improves quality(Gortler96)24Rendering a Light FieldWithout usinggeometryUsing approximategeometry(Gortler96)525Other IBR Representations•Texturemaps• Panorama•Etc.(McMillan)Texture maps are anIBR representation!26QuickTime VR• Cylindrical panorama at certain pointsCylindrical Panorama27Sea of Images• My current work with Dan AliagaMultiresolutionData StructureFloor plan with sampledviewpoints28IBR Summary• Advantageso Photorealistic - by definitiono Do not have to create 3D detailed modelo Do not have to do lighting simulationo Performance independent of scene• Disadvantageso Static scenes onlyo Real-world scenes onlyo Difficult for scenes with specularities, etc.o Limited range of viewpointso Limited
View Full Document
Unlocking...