COMP 790COMP 790--096: Computational Photography096: Computational PhotographyBasic InfoBasic Info• Instructor: Svetlana Lazebnik ([email protected])• Office hours: By appointment, FB 244• Class webpage:http://www.cs.unc.edu/~lazebnik/fall08TodayToday• What is computational photography?• Course overview• Course requirementsWhat is Computational Photography?What is Computational Photography?• Definition 1: the use of photographic imagery to create graphics contentTraditional Computer Graphics3D geometryphysicssimulationprojectionSlide credit: A. EfrosState of the Art•Amazingly real•But so sterile, lifeless, futuristicSlide credit: A. EfrosThe richness of our everyday worldPavia, ItalySlide credit: A. EfrosBeauty in complexityBlue Mountains, AustraliaSlide credit: A. EfrosWhich parts are hard to model?Slide credit: A. EfrosPeopleFrom “Final Fantasy”On the Tube, LondonSlide credit: A. EfrosFaces / HairPhoto by Joaquin Rosales GomezFrom “Final Fantasy”Slide credit: A. EfrosUrban ScenesVirtual LA (SGI)Photo of l LASlide credit: A. EfrosNatureRiver Cherwell, OxfordSlide credit: A. EfrosThe Realism Spectrum+ easy to create new worlds+ easy to manipulate objects/ viewpoint- very hard to look realistic+ instantly realistic+ easy to aquire- very hard to manipulate objects/viewpointComputer Graphics PhotographyComputationalPhotographyRealismManipulationEase of captureSlide credit: A. EfrosWhat is Computational Photography?What is Computational Photography?• Definition 1: the use of photographic imagery to create graphics content• Definition 2: The use of computational techniques to overcome limitations of conventional photographyLimitations of traditional photographyLimitations of traditional photography• Blur, camera shake, noise, damageLimitations of traditional photographyLimitations of traditional photography• Limited resolutionLimitations of traditional photographyLimitations of traditional photography• Bad color / no colorLimitations of traditional photographyLimitations of traditional photography• Unwanted objectsLimitations of traditional photographyLimitations of traditional photography• Unfortunate expressionsLimitations of traditional photographyLimitations of traditional photography• Limited dynamic rangeLimitations of traditional photographyLimitations of traditional photography• Single viewpoint, static 2D pictureLimitations of traditional photographyLimitations of traditional photography• Single depth of focusCourse overviewCourse overview• Data-driven image synthesis• Image enhancement and restoration– Removing camera shake and motion blur– Color image manipulation– Image completion• Combining multiple images– Compositing – Panoramas, mosaics, collages– Warping and morphing• From 2D to 3D– Single-view reconstruction– Multiple-view reconstruction• Beyond conventional cameras– High dynamic range imaging – Light field cameras, coded apertures • Digital image forensicsDataData--driven image synthesisdriven image synthesisTexture synthesisEfros & Leung (1999), Efros & Freeman (2001)Image analogiesHertzmann et al. (2001)Image super-resolutionRemoving Camera Shake and Motion BlurRemoving Camera Shake and Motion BlurFergus et al. (2006) Levin (2006)Yuan et al. (2007)Color image manipulationColor image manipulationColorizationLevin et al. (2004)Color HarmonizationCohen-Or et al. (2006)White Balance AdjustmentHsu et al. (2008)Image completionImage completionImage completion using millions of photographsHays and Efros (2007)InpaintingBertalmio et al. (2000)Image completion with structure propagationSun et al. (2005)Image compositingImage compositingInteractive Digital PhotomontageAgarwala et al. (2004)Face SwappingBitouk et al. (2008)Panoramas, collages, etc.Panoramas, collages, etc.Panorama stitching AutoCollageRother et al. (2006)Multi-viewpoint panoramasAgarwala et al. (2006)Warping and morphingWarping and morphingImage deformation using moving least squaresSchaefer et al. (2006)Face morphingFrom 2D to 3DFrom 2D to 3DPhoto TourismSnavely et al. (2006)Photo Pop-upHoiem et al. (2005)Modeling and Rendering Architecture from PhotographsDebevec et al. (1996)Beyond conventional camerasBeyond conventional camerasHigh Dynamic Range ImagingKopf et al. (2007)Gigapixel ImagesNg et al. (2005)Light Field PhotographyImage ForensicsImage Forensics• How to detect manipulated images?What will not be coveredWhat will not be covered• Advanced image-based modeling and rendering– View interpolation– Relighting– Combining photographic imagery with standard graphics imageryCourse requirementsCourse requirements• Paper presentation: 30%– Think of yourself as “professor” for one class– Make sure you understand all the important techniques in the paper– Integrate related papers, other material covered in the course– Try to encourage discussion• Participation: 20% – Read all papers– Come to class– Ask and answer questions– Talk to me about your presentation or project• Final project: 50% – Implementation– Grading is based primarily on effort and initiative – so try to be creative!– Brief project presentation at the end of the courseProject IdeasProject Ideas• “Straightforward” ideas: implement texture synthesis, panorama stitching, etc.Texture transferTexture transferTour into the PictureTour into the PictureHorry et al. (1997)Creating Creating ““JoinersJoiners””David HockneyFlickr“Hockneyesque”poolL. Zelnik-Manor and P. Perona, Automating Joiners, 5th International Symposium on Non-Photorealistic Animation and Rendering, 2007Face MorphingFace MorphingPlaying around with faces on Playing around with faces on hotornot.comhotornot.comB. Davis and S. Lazebnik, “Analysis of Human Attractiveness Using ManifoldKernel Regression,” ICIP 2008Face beautificationFace beautificationT. Leyvand, D. Cohen-Or, G. Dror and D. Lischinski, Data-Driven Enhancement of Facial Attractiveness, SIGGRAPH 2008Hybrid imagesHybrid imagesA. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,”SIGGRAPH 2006Background ReplacementBackground ReplacementPreliminary results by Sashi Kumar Penta““Infinite PanoramasInfinite Panoramas””J. Sivic, B. Kaneva, A. Torralba, S. Avidan, and W. Freeman,Creating and Exploring a Large Photorealistic Virtual Space,Internet Vision Workshop, 2008Creating unlikely juxtapositionsCreating unlikely juxtapositionsCreating unlikely juxtapositionsCreating unlikely