Slide 1Two presentations today:Slide 3From Pixels to PerceptionSlide 5Perceptual OrganizationKey Research Questions in Perceptual OrganizationSlide 8Contours and junctions are fundamental…Some computer vision history…However in the 90s …At Berkeley, we took a contrary view…Slide 13Slide 14Contour detection ~1970Contour detection ~1990Contour detection ~2004Contour detection ~2008 (gray)Contour detection ~2008 (color)OutlineSlide 21Slide 22Martin, Fowlkes, Malik PAMI 04Individual FeaturesSlide 25OutlineSlide 27Slide 28Eigenvectors carry contour informationSlide 30Slide 31Slide 32Comparison to other approachesSlide 34OutlineSlide 36Slide 37Slide 38Slide 39Better object recognition using previous version of PbOutlinePower laws for contour lengthsSlide 43Slide 44Slide 45Slide 46Ren, Fowlkes, Malik ECCV ‘06Forty years of contour detectionForty years of contour detectionCurvilinear GroupingSlide 51Slide 52Computational PhotographyReadingsSourcesSourcesSlide 57Panography -Panography -PanographySlide 61Problem: Dynamic RangeProblem: Dynamic RangeVarying ExposureHDR images — multiple inputsHDR images — mergedCamera is not a photometer!Slide 68Camera CalibrationCamera sensing pipelineCamera sensing pipelineSlide 72Ways to vary exposureShutter SpeedShutter SpeedThe AlgorithmResponse CurveThe MathMatLab codeResults: digital cameraReconstructed Radiance MapResults: Color FilmRecovered Response CurvesThe Radiance MapThe Radiance MapPortable FloatMap (.pfm)Radiance Format (.pic, .hdr)ILM’s OpenEXR (.exr)Slide 89HDR images — mergedWhat about scene motion?With motion compensationRegistration (global)Registration (local)Now What?Slide 96Tone MappingSimple Global OperatorGlobal Operator (Reinhart et al)Global Operator ResultsSlide 101What do we see?What does the eye sees?MetamoresSlide 105Naïve: Gamma compressionGamma compression on intensityOppenheim 1968, Chiu et al. 1993HalosOur approachBilateral filterStart with Gaussian filteringBilateral filtering is non-linearOther viewContrast reductionDynamic range reductionSummary of approachSlide 118Gradient Tone MappingGradient attenuationSlide 121Tonal ManipulationInterpretation 1:Interpretation 2:Interpretation 3:This Work is About:Existing ToolsTone Reproduction OperatorsAutomatic vs. InteractiveAutomatic vs. InteractiveBut What About Photoshop?ExampleApproachUser interfaceInput: constraintsResult: adjustment mapConstraint PropagationInfluence FunctionsInfluence FunctionsAutomatic InitializationResults – Automatic modeResults – Automatic ModeResults – Automatic modeSlide 144Slide 145Slide 146Snapshot EnhancementSnapshot EnhancementSpatially Variant White BalanceComparison of tone mappersSlide 151Slide 152Flash + non-flash imagesFlash + non-flash imagesJoint bilateral filterBilateral detail filterFinal resultSlide 158CodaC280, Computer VisionProf. Trevor [email protected] 23: Segmentation II & Computational Photography TeaserTwo presentations today:3 Contours and Junctions in Natural ImagesJitendra MalikUniversity of California at Berkeley(with Jianbo Shi, Thomas Leung, Serge Belongie, Charless Fowlkes, David Martin, Xiaofeng Ren, Michael Maire, Pablo Arbelaez)4From Pixels to PerceptionTigerGrassWaterSandoutdoorwildlifeTigertaileyelegsheadbackshadowmouth5I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.---- Max Wertheimer, 19236Perceptual OrganizationGroupingFigure/Ground7Key Research Questions in Perceptual Organization •Predictive power–Factors for complex, natural stimuli ?–How do they interact ?•Functional significance–Why should these be useful or confer some evolutionary advantage to a visual organism?•Brain mechanisms–How are these factors implemented given what we know about V1 and higher visual areas?8Attneave’s Cat (1954)Line drawings convey most of the information9Contours and junctions are fundamental…•Key to recognition, inference of 3D scene properties, visually- guided manipulation and locomotion…•This goes beyond local, V1-like, edge-detection. Contours are the result of perceptual organization, grouping and figure/ground processing10Some computer vision history…•Local Edge Detection was much studied in the 1970s and early 80s (Sobel, Rosenfeld, Binford-Horn, Marr-Hildreth, Canny …)•Edge linking exploiting curvilinear continuity was studied as well (Rosenfeld, Zucker, Horn, Ullman …)•In the 1980s, several authors argued for perceptual organization as a precursor to recognition (Binford, Witkin and Tennebaum, Lowe, Jacobs …)11However in the 90s …1. We realized that there was more to images than edges•Biologically inspired filtering approaches (Bergen & Adelson, Malik & Perona..)•Pixel based representations for recognition (Turk & Pentland, Murase & Nayar, LeCun …)2. We lost faith in the ability of bottom-up vision•Do minimal bottom up processing , e.g. tiled orientation histograms don’t even assume that linked contours or junctions can be extracted•Matching with memory of previously seen objects then becomes the primary engine for parsing an image.√?12At Berkeley, we took a contrary view…1. Collect Data Set of Human segmented images2. Learn Local Boundary Model for combining brightness, color and texture3. Global framework to capture closure, continuity4. Detect and localize junctions5. Integrate low, mid and high-level information for grouping and figure-ground segmentation13D. Martin, C. Fowlkes, D. Tal, J. Malik. "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics", ICCV, 2001Berkeley Segmentation DataSet [BSDS]1415Contour detection ~19701516Contour detection ~19901617Contour detection ~20041718Contour detection ~2008 (gray)1819Contour detection ~2008 (color)1920Outline1. Collect Data Set of Human segmented images2. Learn Local Boundary Model for combining brightness, color and texture3. Global framework to capture closure, continuity4. Detect and localize junctions5. Integrate low, mid and high-level information for grouping and figure-ground segmentation21Contours can be defined by any of a number of cues (P. Cavanagh)22Grill-Spector et al. , Neuron 1998Objects from disparityObjects from texture Objects from luminanceCue-Invariant RepresentationsLine drawingsGray level photographsObjects from motion23Martin,