High Dynamic Range (HDR) Imaging Why and How ?March 31 2011March 31, 2011Suat U. AyAittPfAssistant ProfessorUniversity of IdahoElectrical and Computer EngineeringOutline• Introduction to imaging devicesWh t i hi h d i (HDR) i i ?•What is high dynamic range (HDR) imaging?• Why do we need high dynamic range (HDR) imaging?• How to achieve high dynamic range imaging?•A solutions for CMOS image sensors•A solutions for CMOS image sensors– The hybrid CMOS pixel technology• Conclusion and future directions203/31/2011 – Research Colloquium @ U of Idaho - MoscowImaging DevicesLight SourceOpticsImage SensorSceneDSPVideoASPADCDASP : Analog Signal ProcessorPixel ArrayTiming & ControlPhotons' trajectory3ASP : Analog Signal Processor ADC : Analog to Digital ConverterDSP : Digital Signal ProcessorPixel Array03/31/2011 – Research Colloquium @ U of Idaho - MoscowCMOS Imaging System on a ChipCMOS Active Pixel Color Imaging ArrayDigital Logic for• User Interface•Sensor SetupSensor Setup• Timing Generator• Digital SignalProcessingCl P i–Color Processing–White Balance–Image Enhancement• Data Output pFormattingAnalog SignalAnalog Signal Processing• Data Sampling• Noise ReductionGi4•GainAnalog-To-Digital Conversion03/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is High Dynamic Range (HDR) Imaging?503/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time603/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time– Like what?703/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time–Like: Driving in and out of a tunnel (Auto vision)Image captured by an gp yHDR ImagerImage captured by a Regular Imager8Regular Imager03/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time–Like:Machine vision applicationsImage captured by aItdb9Image captured by a Regular ImagerImage captured by an HDR Imager03/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time–Like: Scientific applicationsImage captured by an HDR ImagerHDR Imager1003/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time–Like: Everyday imaging11Image captured by a Regular ImagerImage captured by an HDR Imager03/31/2011 – Research Colloquium @ U of Idaho - MoscowWhat is dynamic range for imagers anyway?• Dynamic range of an imaging device–Ability to see details of dark and bright spots/areas on ygpa captured scene at the same time–Like: Everyday imagingLow gainHigh gaingain12Images captured by a Regular ImagersImage captured by an HDR Imager03/31/2011 – Research Colloquium @ U of Idaho - MoscowDo we know any HDR imaging technologies around us?1303/31/2011 – Research Colloquium @ U of Idaho - MoscowHDR imaging technologies•Our given eyes and seeing brain gy g– Human Visual System (HVS)• 5-6 order of dynamic range ~120dB–Logarithmic and adaptive response1403/31/2011 – Research Colloquium @ U of Idaho - MoscowHDR imaging technologies• Photographic filmgp• 4-5 order of dynamic range ~90-100dB– Nonlinear and fixed response1503/31/2011 – Research Colloquium @ U of Idaho - MoscowHDR imaging technologies• Semiconductor imagersg– CCD or CMOS image sensors• 2-3 order of dynamic range ~40-60dB–Linear and fixed response1603/31/2011 – Research Colloquium @ U of Idaho - MoscowHDR imaging technologies• Current HDR imagersg– Heavy signal processing to achieve HDR (= slow, high power)• 7-8 order of dynamic range ~160dB–Logarithmic and fixed response1703/31/2011 – Research Colloquium @ U of Idaho - MoscowHDR Imaging: How?HDR Imaging: How?1803/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?Standard (linear)ΔVPIXVenusihSunSun(max)Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. Venus()Light19(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?ΔVPIXStandard (linear)Venus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 20(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?ΔVPIXStandard (linear)Venus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 21(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?ΔVPIXStandard (linear)Venus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 22(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?Now what ?ΔVPIXStandard (linear)Venus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 23(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?Introduce nonlinearityHigh Dynamic RangeΔVPIXyVenus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 24(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging! How?More..ΔVPIXHigh Dynamic RangeVenus1010SunSun(max)Venus()Venus at the Edge Credit: D. Kiselman, et al. (Inst. for Solar Physics), Royal Swedish Academy of Sciences. 25(max)03/31/2011 – Research Colloquium @ U of Idaho - MoscowHigh Dynamic Range Imaging!