CS 414 – Multimedia Systems Design Lecture 12 – MPEG-4 and H.264 (Part 7) Administrative Outline MPEG-4 ExampleCompositionVideo Syntax StructureSpatial Scalability Examples of Base and Enhancement LayersTemporal ScalabilityHigh Level Codec for Generalized ScalabilityComposition (cont.)MPEG-4 Part 11 – Scene DescriptionMPEG-4 Rendering (Composition at Decoder)Integration and Synchronization of Multiple StreamsInteraction as ObjectsHierarchical ModelSynchronizationMPEG-4 parts MPEG-4 AudioH.26XH.263 – PB-Frames ModeComment on Temporal Scalability H.264/MPEG-4 AVC Part 10H.264 CharacteristicsH.264 CharacteristicsH.264 CharacteristicsH.264 Characteristics ConclusionCS 414 - Spring 2011CS 414 – Multimedia Systems DesignLecture 12 – MPEG-4 and H.264 (Part 7)Klara NahrstedtSpring 2011CS 414 - Spring 2011Administrative MP1 – deadline February 18 Homework 1 – posted February 21 Watch for Android tutorials See posting on newsgroup Organized by ACM SIGSoftOutline MPEG-4 Reading: Media Coding book, Section 7.7.2 – 7.7.5 http://www.itu.int/ITU-D/tech/digital-broadcasting/kiev/References/mpeg-4.html http://en.wikipedia.org/wiki/H.264 Available software Xvid – free software http://www.bing.com/videos/watch/video/xvid-free-download-mpeg-4-video-codec-for-pcCS 414 - Spring 2011MPEG-4 ExampleISO N3536 MPEG4 CS 414 - Spring 2011CompositionScenefurnituredesk globecharactervoicespriteISO N3536 MPEG4 CS 414 - Spring 2011(Video Object VO0)(Video Object VO1)Video Syntax StructureCS 414 - Spring 2011New MPEG-4 Aspect: Object-based layered syntactic structure (Base Layer)(Enhancement Layer 1)(I-VOP) (P-VOP) (P-VOP) (B-VOP)Spatial Scalability CS 414 - Spring 2011VOP which is temporally coincident with I-VOP in the base layer, is encoded as P-VOP in the enhancement layer. VOP which is temporally coincident with P-VOP in the base layer is encoded as B-VOP in the enhancement layer. I-VOP P-VOPP-VOPP-VOP(e.g., VOL0 for VO0)(e.g., VOL1 for VO0)(B-VOP) (B-VOP)Examples of Base and Enhancement LayersCS 414 - Spring 2011Temporal ScalabilityCS 414 - Spring 2011High Level Codec for Generalized ScalabilityCS 414 - Spring 2011Composition (cont.) Encode objects in separate channels encode using most efficient mechanism transmit each object in a separate stream Composition takes place at the decoder,rather than at the encoder requires a binary scene description (BIFS) BIFS is low-level language for describing: hierarchical, spatial, and temporal relationsCS 414 - Spring 2011MPEG-4 Part 11 – Scene Description BIFS – Binary Format for Scenes Coded representation of the spatio-temporal positioning of audio-visual objects as well as their behavior in response to interactions Coded representation of synthetic 2D and 3D objects that can be manifested audibly and/or visibly BIFS – MPEG-4 scene description protocol to Compose MPEG-4 objects Describe interaction about MPEG-4 objects Animate MPEG-4 objects BIFS Language Framework – XMT (textual representation of multimedia content using XML) Accommodates SMIL, W3C scalable vector graphics and VRML (now X3D)CS 414 - Spring 2011MPEG-4 Rendering (Composition at Decoder)ISO N3536 MPEG4 CS 414 - Spring 2011Integration and Synchronization of Multiple StreamsCS 414 - Spring 2011Source: http://mpeg.chiariglione.org/technologies/mpeg-4/mp04-bifs/index.htmInteraction as Objects Change colors of objects Toggle visibility of objects Navigate to different content sections Select from multiple camera views change current camera angle Standardizes content and interaction e.g., broadcast HDTV and stored DVDCS 414 - Spring 2011Hierarchical Model Each MPEG-4 movie composed of tracks each track composed of media elements (one reserved for BIFS information) each media element is an object each object is a audio, video, sprite, etc. Each object specifies its: spatial information relative to a parent temporal information relative to global timelineCS 414 - Spring 2011Synchronization Global timeline (high-resolution units) e.g., 600 units/sec Each continuous track specifies relation e.g., if a video is 30 fps, then a frame should be displayed every 33 ms. Others specify start/end timeCS 414 - Spring 2011MPEG-4 parts MPEG-4 part 2 Includes Advanced Simple Profile, used by codecs such as Quicktime 6 MPEG-4 part 10 MPEG-4 AVC/H.264 also called Advanced Video Coding Used by coders such as Quicktime 7 Used by high-definition video media like Blu-ray DiscCS 414 - Spring 2011MPEG-4 Audio Bit-rate 2-64kbps Scalable for variable rates MPEG-4 defines set of coders Parametric Coding Techniques: low bit-rate 2-6kbps, 8kHz sampling frequency Code Excited Linear Prediction: medium bit-rates 6-24 kbps, 8 and 16 kHz sampling rate Time Frequency Techniques: high quality audio 16 kbps and higher bit-rates, sampling rate > 7 kHz CS 414 - Spring 2011H.26X H.261 – CCITT Recommendation of ITU-T Standard Developed for interactive conferencing applications Symmetric coder - real-time encoding and decoding Rates of p x 64 Kbps for ISDN networks Only I and P frames H.263 – established 1996 Used for low bit rate transmission Improvements of error correction and performance Takes in PB-frames mode Temporal, Spatial and SNR scalabilityCS 414 - Spring 2011H.263 – PB-Frames Mode A PB-frames consist of two pictures encoded as one unit. PB-frame consists of One P-picture which is predicted from last decoded P-picture One B-picture which is predicted from last decoded P-picture and the P-picture currently being decoded. CS 414 - Spring 2011PB PPB-framesDecoded P-picture Current P-pictureComment on Temporal Scalability Temporal scalability is achieved using B-pictures These B pictures differ from B-picture in PB-frames they are not syntactically intermixed with subsequent P-picture H.263 is used for low frame rate apps (e.g., mobile), hence in base layer there is one B-picture between I and P pictures. CS 414 - Spring 2011I B PBPH.264/MPEG-4 AVC Part 10 Joint effort between ITU- Video Coding Experts Group (VCEG) and ISO/IEC Moving Picture Experts Group (MPEG) 2003 completed H.264 – codec Standard for Blu-ray Discs Streaming internet standard for videos on YouTube and iTunes Store web software Adobe Flash Player and
View Full Document
Unlocking...