Video Compression and Streaming over the InternetOutlinePart I Error-resilient Scalable Video CompressionWhy Scalable Video – Rate ControlRelated Work – Scalable VideoError-resilient Scalable VideoPart II TCP-friendly Rate ControlOverviewTCP-Friendly Rate ControlSlide 23Streamed Video From HK - BerkeleyPart III The Layered FEC Approach to Multicast Error-controlSlide 29Related WorkLayered FECLFEC – Meeting needs of viewersLFEC – AdvantagesImplementing Layered FECCreating menu of Data+FECOptimal protection at 5% lossOptimal protection at 35% lossSlide 41Overview: Multicast Rate ControlEquation-based Rate ControlSimulation Setup - DumbbellRLM+TCPRLC (Exp)+TCPERC+TCPFairness FunctionResults – Fairness ComparisonMbone ExperimentSlide 52Visually…Part IV Streaming MPEG Video over DiffServ NetworksVideo + DiffServScheme OverviewSimple Delay DifferentiationRefined Delay DifferentiationMPEG: Delay Spread (simple)MPEG: Delay Spread (refined)MPEG: Loss DifferentiationLoss Spread…Slide 63Slide 64Packet ClassifiersLoss-optimized ClassifiersDelay-optimized ClassifiersSimulation SetupSimulation Results - DistortionDelay Results – light loadDelay Results – heavy loadSlide 72What next…Wai-tian TanWai-tian TanVideo and Image Processing LabVideo and Image Processing LabUC BerkeleyUC BerkeleySeptember, 2000September, 2000Video Compression and Video Compression and Streaming over the InternetStreaming over the Internet2OutlineOutlineBest-effort Internet–Error-resilient, scalable compression–Unicast: TCP-friendly transport–Multicast: Layered FECFuture Internet–MPEG-2 streaming over DiffServ networks3Part IPart IError-resilient Scalable Video Error-resilient Scalable Video CompressionCompression4Why Scalable Video – Rate ControlWhy Scalable Video – Rate ControlTimeRateTimeRateSourceReceiver 1Receiver 2125Related Work – Scalable VideoRelated Work – Scalable VideoMany existing approaches–Scalable MPEG-2, H.263–MPEG-4 FGS proposal–Taubman 94–And many others…Focus: best rate-distortion under ZERO loss6Error-resilient Scalable VideoError-resilient Scalable Video3D subband codingData partitioning in the coefficient/subband domainLow-complexity inter-bit-plane codingIndependent decodableEqual visual importanceDispersive error losses19Part IIPart IITCP-friendly Rate ControlTCP-friendly Rate Control20OverviewOverviewRate control–Find “appropriate” rate–AdaptabilityFairness–Globally fair–TCP-friendly21TCP-Friendly Rate ControlTCP-Friendly Rate ControlprttMTUT22.1TCP + Video–Retransmission delay–Large variation in throughputPurpose: –share bandwidth fairly with TCP on a macroscopic scaleMathis et. al. [1996, 1997], Floyd et. al. [1997]. Average TCP throughput:23TCP-Friendly ProtocolTCP-Friendly ProtocolHK - BerkeleyToronto - Berkeley24Streamed Video From HK - BerkeleyStreamed Video From HK - BerkeleyChannel Loss Rate between 0 and 23%average 3.8%Average MSE 10528Part IIIPart IIIThe Layered FEC Approach to The Layered FEC Approach to Multicast Error-controlMulticast Error-control29OverviewOverviewChallenges: Heterogeneity.–Bandwidth: scalable video + layered multicast.–Packet loss rate.Goal: Unified framework for multicast rate and error control.30Related WorkRelated WorkFEC Approaches–Vicisano et al. [1997]–Digital Fountain [1998]–Pejhan et al. [1996], Nonnenmacher et al [1998]–Chou [2000]Other Approaches–SRM [1995]–RMTP [1997]31Layered FECLayered FECFEC Layer 1FEC Layer 2Time = 0GOP 0-1GOP 0-2GOP 0-3Time = GOP 1-1GOP 1-2GOP 1-3FEC 0-1Time = 2GOP 2-1GOP 2-2GOP 2-3FEC 0-2FEC 1-1Time = 3GOP 3-1GOP 3-2GOP 3-3FEC 1-2FEC 2-1Video Layer 1Video Layer 2Video Layer 332LFEC LFEC – Meeting needs of viewers– Meeting needs of viewers SourceInteractive viewerPassive viewer, no lossPassive viewer, low lossPassive viewer, high lossVideo DataFEC layer 1FEC layer 2332133LFEC LFEC – Advantages– AdvantagesFlexible–Allow receivers to adjust reliability according to experienced loss rate and latency requirementsEfficient–FEC layers only reach receivers that need them–Multicast of FEC and data layers Architecturally simple –Requires only IP-multicast–No complex architecture for multicast retransmissionsScalable video:–Unequal error protection –No overall data expansion34Implementing Layered FECImplementing Layered FECSource: –Create menu of Data and FEC layers–Generate FEC layersSink:–Determine available rate (budget)–Optimize within menu given budget38Creating menu of Data+FECCreating menu of Data+FECTotal ordering for FEC stream?–Pro: simplicity–Con: less flexible39Optimal protection at 5% lossOptimal protection at 5% loss01002003004005006007008009001000Rate (kbps)FEC-7FEC-6FEC-5FEC-4FEC-3FEC-2FEC-1FEC-0Data40Optimal protection at 35% lossOptimal protection at 35% loss01002003004005006007008009001000Rate (kbps)FEC-2FEC-1FEC-0Data41Creating menu of Data+FECCreating menu of Data+FECEach FEC layer protects one data layerMultiple FEC layers can protect same data layerOrdering in FEC layers that protect same data layer42Overview: Multicast Rate ControlOverview: Multicast Rate ControlExisting schemes–No loss: probe for more bandwidth–Loss: drop layer–RLM [1996]•explicit intra-session coordination–RLC [1997]•implicit intra-session coordinationNo inter-session coordination43Equation-based Rate ControlEquation-based Rate ControlWhat: Rate as function of packet loss rateWhy:–No probing, compute bandwidth directly–Common side information•Links through same bottleneck sees same loss–Use common information to infer bandwidth allows fair sharing44Simulation SetupSimulation Setup - Dumbbell - DumbbellSender 1Sender 4Receiver 1Receiver 4100Mbps2ms2 Mbps40ms45RLM+TCPRLM+TCP0 100 200 300 400 500 600 700 800 900 100005001000Time (s)Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)46RLC (Exp)+TCPRLC (Exp)+TCP0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600 700 800 900 100005001000Tims (s)Rate (kbps)47ERC+TCPERC+TCP0 100 200 300 400 500 600 700 800 900 100005001000Rate (kbps)0 100 200 300 400 500 600