Matting and TransparencyHow does Superman fly?“Pulling a Matte”Average/Median ImageBackground SubtractionCrowd Synthesis (with Pooja Nath)Background SubtractionBlue ScreenBlue Screen mattingThe UltimatteBlue screen for superman?Semi-transparent mattesReview: two issuesReview: alpha channelMatting Problem: Mathematical DefinitionWhy is general matting hard?Solution #1: No Blue!Solution #2: Gray or FleshTriangulation Matting (Smith & Blinn)The AlgorithmTriangulation Matting ExamplesMore ExamplesMore examplesProblems with MattingSlide Number 25Environment Matting EquationExplanation of FEnvironment MattesPerformanceHow much better is Environment Matting?How much better is Environment Matting?Movies!Fast Separation of Direct and Global Images Using High Frequency IlluminationSlide Number 34Slide Number 35Slide Number 36Slide Number 37Slide Number 38Slide Number 39Slide Number 40Slide Number 41Slide Number 42Slide Number 43Slide Number 44Slide Number 45Slide Number 46Slide Number 47Slide Number 48Slide Number 49Slide Number 50Slide Number 51Slide Number 52Matting and Transparency15-463: Computational PhotographyAlexei Efros, CMU, Fall 2007How does Superman fly?Super-human powers?ORImage Matting?“Pulling a Matte”Problem Definition:• The separation of an image C into– A foreground object image Co , – a background image Cb , – and an alpha matte α• Co and α can then be used to composite the foreground object into a different imageHard problem• Even if alpha is binary, this is hard to do automatically (background subtraction problem)• For movies/TV, manual segmentation of each frame is infeasible • Need to make a simplifying assumption…Average/Median ImageWhat can we do with this?Background Subtraction--==Crowd Synthesis (with Pooja Nath)1. Do background subtraction in each frame2. Find and record “blobs”3. For synthesis, randomly sample the blobs, taking care not to overlap themBackground SubtractionA largely unsolved problem…EstimatedbackgroundDifference Image ThresholdedForegroundon blueOne videoframeBlue ScreenBlue Screen mattingMost common form of matting in TV studios & moviesPetros Vlahos invented blue screen matting in the 50s. His Ultimatte® is still the most popular equipment. He won an Oscar for lifetime achievement.A form of background subtraction:• Need a known background• Compute alpha as SSD(C,Cb) > threshold– Or use Vlahos’ formula: α = 1-p1 (B-p2 G)• Hope that foreground object doesn’t look like background – no blue ties!• Why blue?• Why uniform?The Ultimattep1 and p2Blue screen for superman?Semi-transparent mattesWhat we really want is to obtain a true alpha matte, which involves semi-transparency• Alpha between 0 and 1Review: two issuesSemi-transparent objectsPixels too largeReview: alpha channelAdd one more channel:• Image(R,G,B,alpha) Sprite!Encodes transparency (or pixel coverage):• Alpha = 1: opaque object (complete coverage)• Alpha = 0: transparent object (no coverage)• 0<Alpha<1: semi-transparent (partial coverage)Example: alpha = 0.7Partial coverage or semi-transparencyMatting Problem: Mathematical DefinitionWhy is general matting hard?Solution #1: No Blue!Solution #2: Gray or FleshTriangulation Matting (Smith & Blinn)How many equations?How many unknowns?Does the background need to be constant color?The AlgorithmTriangulation Matting ExamplesMore ExamplesMore examplesProblems with MattingImages do not look realisticLack of Refracted LightLack of Reflected LightSolution:Modify the Matting EquationEnvironment Matting and Compositingslides by Jay HetlerDouglas E. Zongker ~ Dawn M. Werner ~ Brian Curless ~ David H. SalsinEnvironment Matting EquationC = F + (1- α)B + ΦC ~ ColorF ~ Foreground colorB ~ Background colorα ~ Amount of light that passes through the foregroundΦ~ Contribution of light from Environment that travels through the objectExplanation of ΦR – reflectance imageT – Texture imageEnvironment MattesPerformanceCalibrationMatting: 10-20 minutes extraction time for each texture map (Pentium II 400Mhz)Compositing: 4-40 frames per secondReal-Time?How much better is Environment Matting?Alpha Matte Environment Matte PhotographHow much better is Environment Matting?Alpha Matte Environment Matte PhotographMovies!FastSeparationofDirectandGlobalImages UsingHighFrequencyIlluminationShreeK.NayarGurunandan G.KrishnanColumbiaUniversitySIGGRAPHConferenceBoston,July2006Support:ONR,NSF,MERLMichaelD.GrossbergCityCollegeofNewYorkRamesh RaskarMERLsourcesurfacePDirectandGlobalIlluminationAA:DirectBB:InterrelectionCC:SubsurfaceDparticipatingmediumD:VolumetrictranslucentsurfaceEE:Diffusioncamera],[],[],[ icLicLicLgd+=directglobalradianceDirectandGlobalComponents:Interreflectionssurfaceicamerasource∑=PgjiLjiAicL ],[],[],[jBRDFandgeometryHighFrequencyIlluminationPatternsurfacecamerasourcefractionofactivatedsourceelements],[],[],[ icLicLicLgd+=+αiHighFrequencyIlluminationPatternsurfacefractionofactivatedsourceelementscamerasource],[],[],[ icLicLicLgd+=+α‐],[],[ icLicLg=)1(α−i:21=αmin2LLg=SeparationfromTwoImagesdirect global,minmaxLLLd−=OtherGlobalEffects:SubsurfaceScatteringtranslucentsurfacecamerasourceijOtherGlobalEffects:VolumetricScatteringsurfacecamerasourceparticipatingmediumijDiffuseInterreflectionsSpecularInterreflectionsVolumetricScatteringSubsurfaceScatteringDiffusionSceneSceneDirect GlobalRealWorldExamples:CanYouGuesstheImages?Eggs:DiffuseInterreflectionsDirect GlobalWoodenBlocks:Specular InterreflectionsDirect GlobalKitchenSink:VolumetricScatteringVolumetricScattering:Chandrasekar 50,Ishimaru 78Direct GlobalPeppers:SubsurfaceScatteringDirect GlobalHandDirect GlobalSkin: Hanrahan andKrueger93,Uchida96,Haro 01,Jensenetal.01,Cula andDana02,Igarashietal. 05,Weyrich etal.05Face: WithoutandWithMakeupGlobalDirectGlobalDirectWithoutMakeupWithMakeupBlondeHairHairScattering:Stamm etal.77,BustardandSmith91,Luetal.00Marschner etal.03Direct
View Full Document
Unlocking...