Image-Based RenderingWhy Image-Based Rendering?The need for correspondenceHow to get correspondencesMatchingMosaicsSlide 7Slide 8Slide 9Slide 10Other mosaicing issuesMorphingSlide 13Linear Interpolation of PositionOther InterpolationSlide 16Slide 17Slide 18Slide 19Slide 20InterpolationSlide 22Linear basis for lightingUsing Linear Basis for RenderingReflectance smooths lightingBasis from diffuse lightingSlide 27Non-Photorealistic RenderingDe Carlo and SantellaImage AnalogiesImage-Based RenderingProduce a new image from real images.Combining imagesInterpolationMore exotic methodsWhy Image-Based Rendering?What’s the most realistic image? A photograph.But photographs lack flexibility.Can’t change viewpoint.Can’t change lighting.The need for correspondenceImage-based rendering is mostly combining images to get a new image.Correspondences needed to sensibly combine images.If viewpoint has changed this can be hard.If not, it’s trivial.How to get correspondencesBy hand:Works if few correspondences neededBy matching intensitiesThis is really ~ ½ of computer vision.MatchingSimplest: SSD with windows.Windows needed because pixels not informative enough. Compare windows ==??ffggMostMostpopularpopular• Search for windows that match wellMosaicsTake multiple images and construct one big image.Represented as image, cylinder or sphere.Allows panning and zooming.Simplest kind of motion.•Fixed focal point.Correspondence needed to align images.•Image rectification(Images in paper by Szeliski and Shum, linked to on web page)(Images in paper by Szeliski and Shum, linked to on web page)(Images in paper by Szeliski and Shum, linked to on web page)Other mosaicing issuesPixel interpolation needed.Mosaicing can provide more information at each pixel.Higher resolution images possible.Higher dynamic range.MorphingWhat happens if you interpolate images?Need corresponding points.MorphingCorresponding points needed.Often done by hand.Interpolate each point.Positionand intensity.Also use interpolation for more correspondences.Linear Interpolationof PositionOther InterpolationAlso interpolate intensities.Interpolate to find other point correspondes.Light Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)Light Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)Light Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)Light Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)Light Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)InterpolationWe have possibly non-uniform samples of a 4D space. Must interpolate to fill in.Worry about aliasingLight Field Rendering (Levoy and Hanrahan; paper and slides)(Images from Marc Levoy’s slides: http://graphics.stanford.edu/papers/light/)Linear basis for lightingZ YXSurface normal = (X,Y,Z) albedo = Directional source = (Lx,Ly,Lz)I = (Lx,Ly,Lz)(X,Y,Z) = Lx*X + Ly*Y + Lz*  ZTake Max of this and 0Using Linear Basis for RenderingRender three imagesTake linear combinations.Why can’t we do this with three real images?0 1 2 300.510 1 2 300.511.52lrReflectance smooths lightingBasis from diffuse lighting Z YX2 2 22λ(Z -X -Y )XZ YZ2 2λ(X -Y )XYNote, this can also be done with 9 real images, because this is a basis that contains real imagesIn 3D, real images aren’t in the 3D space, we have to take the max with 0 to get real images.Non-Photorealistic RenderingTake a photo and turn it into a different kind of image.De Carlo and SantellaVideoImage AnalogiesGiven A, A’ and B, generate B’A bit like Efros and Leung texture synthesis.(Pictures from image analogies paper linked to on class web