Low Complexity AVS-China Part-II Video Codec Design using Data-mining TechniquesOutline1 INTRODUCTIONAudio / Video History2. AVS China Video Coding StandardAVS China Part II - ProfilesProfile Feature ComparisonAVS P2 – Layered StructureRGB vs YUVLuminance and ChrominanceInput Video FormatsVideo FormatAVS Codec ArchitecturePicture TypesPredictionIntra PredictionInter PredictionAVS China – Macroblock Type3. Mode DecisionIntra Mode Decisions – Luminance (Y)Intra Mode Decisions - ChrominanceExample – Intra Mode DecisionIntra Mode Decision Example – cont’dInter Mode Decision4. Machine LearningMode decision complexity reductionSlide 27AttributeEdge DetectionEdge Detection Mode DecisionFeature Extraction – an Example5. Summary6. ReferencesReferences – cont’dQ/A ? ~ Thank youLow Complexity AVS-China Part-II Video Codec Design using Data-mining Techniques~ Comprehensive proposalMay 6th, 2010by Jennie G. AbrahamSupervising Professor - Dr K R RaoOutlineIntroductionAVS China Video Coding StandardIdentify area of researchProblem statementProposed solutionSummary1 INTRODUCTION Fig 1.1: Example of a multimedia ecosystemAudio / Video HistoryFig. 1.2: History of audio video coding standards2. AVS China Video Coding StandardAVS China Part 2 is the official video coding standard of China developed by the Audio Video Standard Working Group of China [1].Table 2.1: Different parts of AVS standard [4]AVS China Part II - ProfilesProfile subset of syntax, semantics, and algorithms defined by AVSTable 2.2 Application-based profiles of AVS[11]Profiles Key ApplicationsJizhun (baseline) profileTelevision broadcasting, HDTV, etc.Jiben (basic) profile Mobility applications, etc.Shenzhan (extended) profileVideo surveillance, etc.Jiaqiang (enhanced) profileMultimedia entertainment, etc.Profile Feature ComparisonTable 2.3: Features of the four profiles in AVS-China video codec[11]AVS P2 – Layered StructureFig 2.2 Layered structure of the input data Group of Pictures (GOP)SlicePictureMacroblock8 pixels8 pixelsBlockVideo inputChrominance (Cb,Cr )Luminance (Y)RGB vs YUV YUVRGBUMCP ENEE408G Slides (created by M.Wu & R.Liu © 2002)Luminance and Chrominance4:4:4 formatYCbCrInput Video FormatsVideo FormatFig 2.1 Color formats of input video AVS video codec input formatAVS Codec ArchitectureFigure 2.3: AVS video coding framework[7]+-Picture TypesThree types of pictures are defined by AVS:1. Intra pictures (I-pictures)2. Predicted pictures (P-pictures)3. Interpolated pictures (B-pictures)Prediction1. Intra Prediction (I frames)2. Inter PredictionForward prediction (P frames)Bi-directional prediction (B frames)Intra PredictionFigure 2.6: AVS video coding framework[7] Current block (8 x 8)PredictPredictPreviously coded pixelsPreviously coded pixels*Mode decisionsInter PredictionFigure 2.4: Inter Prediction – forward and backward prediction [19]AVS China – Macroblock Type Fig 2.5 Macroblock type for inter prediction [3]AVS China P2(4 modes)AVS China P7(all 8 modes)3. Mode DecisionComputationally most expensive process in video codingRipe area to investigate complexity reductionFocus area of the current researchConstraints:–Maintain the quality–Maintain the compression ratioIntra Mode Decisions – Luminance (Y)Figure 3.1: Five luminance intra prediction modes[21]Mode 0 - Vertical modeMode 1 – Horizontal modeMode 2 – DC modeMode 3 – Down leftMode 4 – Down rightIntra Mode Decisions - ChrominanceFigure 3.2: Four chrominance intra prediction modes[21]0 DC mode1 Horizontal mode2 Vertical mode3 Plane modeExample – Intra Mode DecisionAlready decodedYet to be decodedBeing currently predictedFigure 3.3: Intra mode decision example[10]Intra Mode Decision Example – cont’dMode 0 Vertical modeSAE - 657Mode 1Horizontal modeSAE - 619Mode 2DC modeSAE - 607Mode 3 Down leftSAE - 200Mode 4Down rightSAE - 1032Being currently predicted*SAE : Sum of Absolute ErrorsLowest SAEBest prediction !Figure 3.4: Intra mode decision based on lowest SAE[10]Inter Mode DecisionMB modes for inter prediction in AVS China P2Each 16x16 MB can have 1 to 4 motion vectors representing itThe MB type that provides the best prediction is chosen after SAE calculation on each !Figure 3.5: Inter mode decision[3]4. Machine LearningData mining Machine learningSupervised learningClassificationDecision tree classifiersC4.5 / J48 algorithmWEKA (Waikato Environment for Knowledge Analysis) data mining tool – open source software, popular suite of machine learning software written in Java, developed at the University of Waikato.Mode decision complexity reductionExtract Attributes(offline)Extract Encoder mode decisions(offline)C4.5(J48) classifierConvert to if-else statementsGenerate decision treesExtract Attributes(online)Replace complex mode decision by if else statements to determine the modeClassify each MB with the correct modeFig 4.1 Low complexity Encoder using machine learningAttributeMean of each current 16x16 MBVariance of each current 16x16 MBMean of 16 4x4 blocks within the current 16x16 MBVariance of this 16 means for the current 16x16 MBVariance of 16 4x4 blocks within the current 16x16 MBMean of each 16x16 residue MBVariance of each 16x16 residue MBMean of 16 4x4 blocks within the 16x16 residue MBVariance of this 16 means for the 16x16 residue MBVariance of 16 4x4 blocks within the 16x16 residue MBEdge detection for 16x16 MB [yes, no]Edge detection for 16x16 residue MB [yes, no]Edge direction for current 16x16 MBEdge direction for 16 4x4 blocks within the current 16x16 MBEdge direction for 16x16 residue MBEdge direction for 16 4x4 blocks within the 16x16 residue MBTotal - ~ 100+ attributes for each macroblockEdge DetectionRoberts PrewittSobelCannyCanny’s edge detection algorithm is computationally more expensive compared to Sobel, Prewitt and Robert’s operator. However, the Canny’s edge detection algorithm performs better than all these operators under almost all scenarios.Quantize edge intensity to 0/1:•set a threshold•white pixel denotes strong edgeEdge Detection Mode DecisionMode 4 Down rightSAE - ?Mode 0 Vertical modeMode 1HorizontalMode 3Down leftMode 4Down rightFeature Extraction – an ExampleFig 4.2 Extracting features of MB adjacent to current block in intra prediction5. SummaryWithin AVS-China Part – II video codec an area for improvement is identified, which is the mode decision computation which is a very
View Full Document