I. INTRODUCTIONA. Packetized elementary streams (PES):II. MULTIPLEXINGIII. De-multiplexingIV. Results[14] MPEG Transport Stream. Link: http://www.iptvdictionary.com/iptv_dictionary_MPEG_Transport_Stream_TS_definition.html[43] B.A. Cipra, “The Ubiquitous Reed-Solomon Codes”. Link: http://www.eccpage.com/reed_solomon_codes.htmlAbstract: Television broadcasting applications such as ATSC-M/H, DVB [16] require that the encoded audio and video streams to be transmitted across a network in a single transport stream containing fixed sized data packets that can be easily recognized and decoded at the receiver. MPEG2 part1 specifies two layers of packetization to achieve a transport stream suitable for digital transmission. In a broadcasting system, multiplexing is a process in which two or more elementary streams are converted into a single transport stream ensuring synchronous playbackof the elementary streams and proper buffer behavior at the decoder. This paper presents a scheme to multiplex the elementary streams of H.264 video and HE AAC v2 audio using the MPEG2 systems specifications [4], then de-multiplex the transport stream and playback the decoded elementary streams with lip synchronization or audio-video synchronization.This paper briefly introduces the MPEG2 systems, two layers of packetization namely program elementary stream (PES) and transportstream (TS), it also introduces the concept of timestamps and the approach followed in this paper and finally the proposed multiplexing and de-multiplexing algorithms, approach followed to achieve synchronization is explainedfollowed finally by the results of the implementation. Index terms: H.264, HEAACv2, multiplexing, MPEG2 systems, PES, TS.I. INTRODUCTIONMobile broadcast systems are becoming increasingly popular as cellular phones and highly efficient digital video compression techniques merge to enable digital TV and multimedia reception on the move. Mobile television broadcast systems like DVB-H (digital video broadcast- handheld) [16] and ATSC-M/H (advanced television systems committee- mobile/handheld) [17] [18] [21] have relatively small bandwidth allocation and the processing power at the target device (decoder) also varies. Hence, the choice of the compression standards used plays an important role. H.264 [1] [5] [6] and HEAACv2 [2] [7] [13] are the codecs used in the proposed method for the video and audio respectively. H.264 [5] is the latest and the most advanced video codec available today. It was jointly developed by the VCEG (video coding experts group) of ITU-T (international telecommunication union) and the MPEG (moving pictures experts group) of ISO/IEC (international standards organization). This standard achieves much greater compression than its predecessors like MPEG-2 video [37], MPEG4 part 2 visual [38] etc. But the higher coding efficiency comes at the cost of increased complexity. The H.264 has been adopted as the video standard for many applications around the world including ATSC [21].HEAACv2 or High efficiency advanced audio codec version 2 also known as enhanced aacplusis a low bit rate audio codec defined in MPEG4 audio profile [2] belonging to the AAC family. ItMultiplexing the elementary streams of H.264 video and MPEG4 HE AAC v2 audio,de-multiplexing and achieving lip synchronizationNaveen Siddaraju and K.R. Rao, IEEE FellowElectrical Engineering Department, University of Texas at Arlington, Arlington, TX Email: [email protected], [email protected] specifically designed for low bit rate applications such as streaming, mobile broadcasting etc. HE AAC v2 has been proven to be the most efficient audio compression tool available today. It comes with a fully featured toolset which enables coding in mono, stereo and multichannel modes (up to 48 channels). HEAACv2 [7] is the adopted standard for ATSC-M/H and many other systems around the world. The encoded bit streams or elementary streams of H.264 and HEAACv2 are arranged as a sequence of access units. An access unit is a coded representation of a frame. Since each frame is coded differently the size of each accessunit also varies. In order to transmit a multimedia content (audio and video) across a channel, the two streams has to be converted in to a single stream of fixed sized packets. For thisthe elementary streams has to undergo two layers of packetization (Fig. 1). The first layer ofpacketization yields Packetized Elementary Stream (PES) and the second layer of packetization where the actual multiplexing takes place results in a stream of fixed sized packets called as Transport Stream (TS). These TS packets are what are actually transmitted across the network using broadcast techniques such as those used in ATSC and DVB [16]. Fig. 1 MPEG2 two layers of packetization [22]A. PACKETIZED ELEMENTARY STREAMS (PES):PES packets are obtained after the first layer of packetization of audio and video coded data. This packetization process is carried out by sequentially separating out the audio and video elementary streams into access units. Hence each PES packet is an encapsulation of one frame of coded data. Each PES packet contains apacket header and the payload data from only one particular stream. PES header contains information which can distinguish between audio and video PES packets. Since the number of bits used to represent a frame in the bit streamvaries (for both audio and video) the size of the PES packets also varies. Figure 2 shows how theelementary stream is converted into PES stream.Fig. 2 Conversion of an elementary stream into PES packets [29]The PES header format used is shown in table 1.The PES header starts with a 3 byte packet start code prefix which is always “0x000001” followed by 1 byte stream id. Stream id is used to uniquely identify a particular stream. Stream id along with start code prefix is known as start code (4 bytes). PES packet length may vary and go up to 65536 bytes. In case of longer elementary stream, the packet length may be set as unbound i.e. 0, only in the case of video stream. The next two bytes in the header is the time stamp field, which contains the playback time information. In the proposed method, framenumber is used to calculate the playback time, which is explained next.Table. 1 PES header format [4]B. TIME STAMP: Time stamps indicate where a particular access unit belongs in time. Audio-video synchronization is obtained by incorporating time stamps into the headers in both video and
View Full Document
Unlocking...