Overview on General Framework & Issues of Image Analysis and Video StreamingOverview and LogisticsRecap: Video Content AnalysisFast Extraction of DC Image From MPEG-1Summary on Video Temporal SegmentationContent-based Image Retrieval (CBIR)Requirements for Various CBIR ApplicationsImage Retrieval from User & System PerspectivesFramework for Forming Image Signature for CBIRImage Similarity Measures for CBIRClustering Methods and Application ScopeVideo CommunicationsMM + Data Comm. = Effective MM Communications?Error-Resilient Coding with Localized Synch MarkerIssues in Video Communications/StreamingData Hiding in Images: An IntroductionExample: Data Embedding by Replacing LSBsExample: LSB Replacement (cont’d)Using Higher LSB Bitplanes for EmbeddingExample: LSB Replacement of Higher BitplanesReview: Pixel DepthEmbedding Basics: Two Simple TriesHow to Improve the Robustness?Distortion from Quantization-based EmbeddingTwo Views of Quantization-based EmbeddingTampering Detection by Pixel-domain Fragile WmkFight Against Forging Tamper-Detection Watermark?Pixel-domain Table-lookup Embedding (Yeung-Mintzer ICIP’97)Yeung’s Fragile Watermark for Tampering DetectionLUT Embedding: Distortion/Security/RobustnessCase Study: Mintzer-Yeung Patent on Fragile WmkReference ReadingsSlide 39A Glimpse at the Patent SystemPatent ProcessStructure of A Utility PatentUseful Contents for Technical StudiesClaims: Crucial Part for Business ValuesFrom Fragile to Robust WatermarkM. Wu: ENEE631 Digital Image Processing (Spring'09)Overview on General Framework & Issues Overview on General Framework & Issues of Image Analysis and Video Streamingof Image Analysis and Video StreamingSpring ’09 Instructor: Min Wu Electrical and Computer Engineering Department, University of Maryland, College Park bb.eng.umd.edu (select ENEE631 S’09) [email protected] Spring’09ENEE631 Spring’09Lecture 20 (4/15/2009)Lecture 20 (4/15/2009)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [2]Overview and LogisticsOverview and LogisticsLast Time: –General methodologies on motion analysis–Video content analysisBasic frameworkTemporal segmentation; Compressed domain processing Today:–Project#2–Overview of content-based image retrieval–A quick guide on video communications–Introduction on data embedding in imagesUMCP ENEE631 Slides (created by M.Wu © 2004)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [3]Recap: Video Content AnalysisRecap: Video Content AnalysisTeach computer to “understand” video content–Define features that computer can learn to measure and compare color (RGB values or other color coordinates) motion (magnitude and directions) shape (contours) texture and patterns–Give example correspondences so that computer can learn build connections between feature & higher-level semantics/concepts statistical classification and recognition techniquesVideo understanding1. Break a video sequence into chunks, each with consistent content ~ “shot”2. Group similar shot into scenes that represent certain events3. Describe connections among scenes via story boards or scene graphs4. Associate shot/scene with representative feature/semantics for future queryUMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [6]Fast Extraction of DC Image From MPEG-1Fast Extraction of DC Image From MPEG-1I frame–Put together DC coeff. from each block (and apply proper scaling)Predictive (P/B) frame–Fast approximation of reference block’s DC –Adding DC of the motion compensation residuerecall DCT is a linear transformSee Yeo-Liu’s paper for more derivations on approximations (DC; DC+2AC)[ ( )] [ ( )] [ ( )]DCT P DCT P DCT Pcur ref diff00 00 00 [ ( )] [ ( )]DCT Ph wDCT Prefi iii00 0014641 234CRUMCP ENEE408G Slides (created by M.Wu © 2002)DC FrameM. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [7]Summary on Video Temporal SegmentationSummary on Video Temporal SegmentationA first step toward video content understanding–Image analysis may be applied to key frame sequences–Motion and temporal info can also be exploitedTwo types of transitions–“Cut” ~ abrupt transition– Gradual transition: Fade out and Fade in; Dissolve; WipeDetecting transitions: can be done on “DC images” w/o full decompression–Detecting cut is relatively easier ~ check frame-wise difference–Detecting dissolve and fade by checking linearity f0 (1 – t/T) + f1 * t/T –Detecting wipe ~ more difficult exploit transition patterns, or linearity of color histogramUMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [8]Content-based Image Retrieval (CBIR)Content-based Image Retrieval (CBIR)An active research area from 1990s with renewed interests –As digital camera/camera-phones become affordable, and image & video sharing services (flickr, YouTube/Google, etc) become popular Reference: a recent comprehensive survey –“Image Retrieval: Ideas, Influences, and Trends of the New Age,” by Datta, Joshi, Li and Wang, ACM Computing Survey, 4/2008Include a broad range of use scenarios and applications–Image browsing, search and retrieval–Automatic image annotation, and related subfieldsWith interests & contributions from multiple fields of study–Multimedia (MM), machine learning (ML), info retrieval (IR), computer vision (CV), and human-computer interaction (HCI)=> See also EE633 pattern recognition and other CS courseM. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [9]Requirements for Various CBIR ApplicationsRequirements for Various CBIR Applications(Fig. from ACM Computing Survey 4/2008 article by Datta, Joshi, Li and Wang)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [10]Image Retrieval from User & System PerspectivesImage Retrieval from User & System Perspectives(Fig. from ACM Computing Survey 4/2008 article by Datta, Joshi, Li and Wang)M. Wu: ENEE631 Digital Image Processing (Spring'09)Lec.20 – Overview on CBIR & Video Comm [11]Framework for Forming Image Signature for CBIRFramework for Forming