Overview of Fine Granularity Scalability in MPEG-4 Video StandardIntroduction - ProblemIntroduction – Video Coding PerformancePrevious Layered Coding MethodsMPEG-2 OverviewTemporal ScalabilityMPEG-2 SNR ScalabilitySpatial ScalabilityMPEG-4 Fine Granularity Scalability TechniqueBitplane Coding TechniqueSlide 11Slide 12Run Level EncodingMPEG-4 Bitplane FGS TechniqueFine Granularity Scalability EncoderFine Granularity Scalability DecoderTruncated BitplaneAdvanced Bitplane TechniquesFrequency Weighting in BitplaneSelectively Enhanced BitplaneOther MethodsSimulation - DCTSimulation - DWTSelectively Enhanced / Weighted Bitplane SetupSimulation – Enhanced Weighted BitplaneSimulation – LiConclusionsQuestions?Overview of Fine Granularity Scalability in MPEG-4 Video StandardWeiping LiPresented by : Brian ErikssonIntroduction - Problem•Tradition System–Video encoded close to channel capacity–Decoded using all of the bits•Internet Video System–Channel capacity changes–Partially decodable within a bit rangeIntroduction – Video Coding PerformanceLow HighChannel BitrateReceived QualityBadGood= Distortion-Rate Curve= Dialup Rate= Broadband Rate= Layered Coding= Desired CodingLow HighChannel BitrateReceived QualityBadGoodPrevious Layered Coding Methods= Layered CodingMPEG-2 Overview•I-Frame – Intracoded Picture–Contains all frame information•P-Frame – Predictive-Coded Picture–Uses previous frame and motion information to reconstruct frame•B-Frame – Bidirectional-Coded Picture–Uses previous frame, next frame, and motion information to reconstruct the frameTemporal Scalability•Video is encoded into two layers with equal resolution but using different prediction.•Base layer has lower frame rateBase Enh. Enh. Base Enh. Enh. Base Enh. Enh.Base X X Base X X Base X XWith Enhancement Layer : Frame Rate = 30 fpsWithout Enhancement Layer : Frame Rate = 10 fpsMPEG-2 SNR ScalabilityEnhancement Layer StreamBase Layer Stream+IDCTVideo Output•Motion prediction includes enhancement information•Efficiency dependent on two factors•Encoder uses enhancement layer (drift can occur)•Decoder receives enhancement layerMotion Compensation+Spatial ScalabilityIDCTEnhancement BitstreamBase Bitstream (Motion Compensated)Upsampler+Enhancement Layer VideoBase Layer Video•Same frame rate, different resolutions•Base frame = 128x128, enhancement = 256x256•Enhancement Layer not in prediction loopLow HighChannel BitrateReceived QualityBadGoodMPEG-4 Fine Granularity Scalability Technique= Desired Coding= Distortion-Rate CurveBitplane Coding TechniqueGiven vector X:X = [-12, -53, 62, -7, 31,…,180,-43,…,5]•Convert to signed magnitude•Separate into sign bit and absolute valuesBitplane Coding TechniqueSign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]Abs(X) = [12, 53, 62, 7,31,…,180,43,…,5]•To find the number of bitplanes needed, find the maximum value of abs(X)•Max(abs(X)) = 180 = 10110100b•8 bitplanes•Sign bit plane •1 bitplaneBitplane Coding TechniquecSign(X) = [0, 0, 1, 0, 1,…,1,0,…,1]X = [-12, -53, 62, -7, 31,…,180,-43,…,5]0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1Run Level Encoding•Symbol : (RUN, EOP)–RUN = Number of Consecutive Zeros before a 1–EOP = 0 if there are more ones–EOP = 1 if the rest of the line are zeros.•Example:–{1,0,1,0,0,0,1,0,0,0,…}•= (0,0),(1,0),(3,1)–{0,0,0,1,0,0,0,…}•= (3,1)MPEG-4 Bitplane FGS Technique•Base layer reaches lower bound of bit-range•Divide image into 8x8 DCT blocks•Divide blocks into Y,U,V color components•Use bitplane run-level coding to encode/decodeFine Granularity Scalability Encoder+-DCT Bitplane EncodingOriginal SignalBase Layer Signal (Motion Compensated)Enhancement Layer SignalFine Granularity Scalability DecoderBitplane DecodingIDCT+Base Layer Video (Motion Compensated)Enhancement Layer VideoTruncated BitplanecNumber of Truncated Layers = 0Received X = [-12, -53, 62, -7,31,…,180,-43,…,-5]Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]0,1,0,1,1,…,0,1,…,10,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1cNumber of Truncated Layers = 1Received X = [-12, -52, 62, -6,30,…,180,-42,…,-4]Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]0,0,0,0,0,…,0,0,…,00,0,1,1,1,…,0,1,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1cNumber of Truncated Layers = 2Received X = [-12, -52, 60, -4,28,…,180,-40,…,-4]Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,1,1,1,1,…,1,0,…,11,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1cNumber of Truncated Layers = 3Received X = [-8, -48, 56, -0,24,…,176,-40,…,-0]Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5]0,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,0,0,…,01,0,1,0,1,…,0,1,…,00,1,1,0,1,…,1,0,…,00,1,1,0,0,…,1,1,…,00,0,0,0,0,…,0,0,…,00,0,0,0,0,…,1,0,…,00,0,1,0,1,…,1,0,…,1Advanced Bitplane TechniquesFrequency Weighting in BitplaneNormal BitplaneDCT IndexFrequency Weighted BitplaneDCT IndexSelectively Enhanced Bitplane•Detect visually significant area•Shift upward in bitplane•More likely to be included in truncated bitstreamOther Methods•Error Resilience in Bitplane•Random burst errors in the bitstream•Resynchronization markers are used to resynch.•Temporal Scalability in Bitplane•Uses FGS to encode/decode the entire temporal enhancement frameSimulation - DCTSimulation - DWTSelectively Enhanced / Weighted Bitplane SetupSimulation – Enhanced Weighted Bitplane1 Layer Removed2 Layers Removed3 Layers Removed4 Layers Removed5 Layers Removed6 Layers RemovedSimulation – LiConclusions •FGS Bitplane method allows for quality parallel to distortion-rate curve•A wavelet-based approach may yield better results•Fairly simple