Page 1Peter A. Steenkiste, SCS, CMU1Lecture 20MultimediaPeter SteenkisteSchool of Computer ScienceDepartment of Electrical and Computer EngineeringCarnegie Mellon University15-441 Networking, Spring 2004http://www.cs.cmu.edu/~prs/15-441Peter A. Steenkiste, SCS, CMU2Lecture Overview•Multimedia Applications • Application Requirements / Challenges• Media Encodings• Case Studies» Streaming Stored Media» Real Time Interactive Media (Voice over IP)• Protocols» Real-Time Transport Protocol (RTP)» Real-Time Transport Control Protocol (RTCP)Peter A. Steenkiste, SCS, CMU3Application Classes•Streaming Stored Media» Pipeline reception of stored files over the network» No need to wait for entire file before viewing» Example: On-demand video • Unidirectional Real-Time Media» Similar, but data is generated on the fly» Non-interactive: just listen/view, no fast-forward!» Example: news broadcast• Interactive Real-Time Media» Example: video conferencePeter A. Steenkiste, SCS, CMU4Streaming Media Applications•Characteristics» Interactive: Users can play, pause» Even rewind/fast-forward• Requirements» Delay: from client request until display start can be 1 to 10 seconds» Time is used to buffer data to reduce chance of stalling» Delay constraints on data delivery to maintain continuous playout.On demand VideoPeter A. Steenkiste, SCS, CMU5Unidirectional Real-Time Applications•Characteristics» By storing media: can pause, rewind» Often used combined with multicast• Requirements» As with streaming media, delay constraints on data delivery to maintain continuous playoutNews BroadcastPeter A. Steenkiste, SCS, CMU6Real-Time Interactive Applications•Requirements» Very stringent delay requirements because of real-time interactive nature» Limit to how much people are willing to adapt to the delay» Video: < 150 msec acceptable (one-way)» Audio: < 150 msec not perceived, < 400 msec acceptable (one-way)Internet TelephonyPage 2Peter A. Steenkiste, SCS, CMU7Roadmap•Multimedia Applications• Application Requirements / Challenges• Media Encodings• Case Studies» Streaming Stored Media» Real Time Interactive Media (Voice over IP)• Protocols» Real Time Protocol (RTP)» Real Time Control Protocol (RTCP)Peter A. Steenkiste, SCS, CMU8Review: Network BasicsSource DestThroughput = Amount of data arriving per unit time.Jitter = Variance of interpacket arrival timesInterpacketArrival TimeDelayLossBest-effort delivery!!Peter A. Steenkiste, SCS, CMU9End-to-End Delay•Important for interactive applications» Video conferencing– Human communication starts to break down at ~500 msec delays» Feedback loops» Delay affected by encoding/decoding• Today’s Internet» Best effort service and end-to-end mechanisms » Provide little opportunity for controlling delay• Longer term (only?) solution: Quality of Service Peter A. Steenkiste, SCS, CMU10Packet Jitter•Real-Time systems must deal with this» Jitter is hard to bound even with QoS• Partial solution: Buffer management» Buffers are used to smooth jitter, but how big should the buffer be?» Unexpected jitter can lead to starvation or buffer overflow if buffer is too small» But over-provisioning buffers increases delayPeter A. Steenkiste, SCS, CMU11Throughput•Estimating throughput is problematic» Media coding may be tunable, but changes in quality are generally perceptually annoying» Throughput will often vary faster than feedback delay which makes estimation hard• Multicast makes this worse• Possible Solutions:» QoS mechanisms» Scalable encoding combined with congestion control (i.e., dynamically adapt compression to bandwidth)Peter A. Steenkiste, SCS, CMU12Packet Loss•Effect of loss is media and encoding specific» Loss of one packet in a video stream may result in the loss of multiple frames» Other packets received can be rendered useless» Retransmission often not feasible (no time)• Possible Solutions:» Encoding that alleviates packet loss effects» Forward Error Correction (FEC)Page 3Peter A. Steenkiste, SCS, CMU13Roadmap•Multimedia Applications• Application Requirements / Challenges• Media Encodings• Case Studies» Streaming Stored Media» Real Time Interactive Media (Voice over IP)• Protocols» Real Time Protocol (RTP)» Real Time Control Protocol (RTCP)Peter A. Steenkiste, SCS, CMU14Media Encoding•Digitize the analogue information.• Sample signal at some frequency» 8KHz, for example• Quantize » represent each sample as some fixed #bits• Compress: make number of bits small» Audio: GSM, G.729, G.723.3, MP3, …» Video: MPEG 1/2/4, H.261, …• Then:» Send -> Receive, decompress/convert, playPeter A. Steenkiste, SCS, CMU15Audio Encoding• Traditional telephone quality encoding» 8KHz samples of 8 bits each Æ 64kbps• CD quality encoding: 44.1 KHz of 16 bits» 1.41 Mbs uncompressed• MP3 compression » Frequency ranges that are divided in blocks, which are converted using DCT, quantized, and encoded12128 kbpsLayer 38192 kbpsLayer 24384 kbpsLayer 1RatioRangePeter A. Steenkiste, SCS, CMU16Image Encoding: JPEG •Divide digitized image in 8 x 8 pixel blocks• The DCT phase converts the pixel block into a block of frequency coefficients» Discrete Cosine Transform – similar to FFT• The quantization phase limits the precision of the frequency coefficient» This is based on a quantization table, which controls the degree of compression• The encoding phase packs this information in a dense fashionPeter A. Steenkiste, SCS, CMU17Video Encoding•Once frames are captured ("raw" video) resulting file is very large:» 320 x 240 x 24-bit color = 230,400 bytes/frame » 15 frames/second = 3,456,000 bytes/second » 10 seconds takes around 30 Mbytes! (no audio) • Commonly-used encodings: AVI, MPEG» Exploit spatial locality inside a single image» Temporal locality across images» Layered encodingPeter A. Steenkiste, SCS, CMU18MPEG Encoding of Video•Three frame types:» I frame: independent encoding of the frame (JPEG)» P frame: encodes difference relative to I-frame (predicted)» B frame: encodes difference relative to interpolated frame» Note that frames will have different sizes• Complex encoding, e.g. motion of pixel blocks, scene changes, ..» Decoding is easier than encoding.• MPEG often uses fixed-rate encoding.I B B P B B P B B I B B P B BPage 4Peter A. Steenkiste, SCS, CMU19MPEG System Streams•Combine one more MPEG video and audio streams in a
View Full Document
Unlocking...