MP2: Video StreamingShu ShiMar 11 2011Thursday, March 10, 2011MP Requirements•Streaming•Video Streaming•Audio Streaming•A/V Sync•TCP and UDP•Pause and Resume•Stats•Bandwidth•Latency•RobustnessThursday, March 10, 2011Streaming -- Protocols•Network Programming Basics•TCP, UDP•bind, listen, connect, accept•send/recv, sendto/recvfrom•Useful Link: •http://beej.us/guide/bgnet/Thursday, March 10, 2011Streaming -- Steps•Connection Establishment•No connection for UDP•Negotiation•Exchange video/audio QoS information•Admission decision•Data Streaming•Replace AVI_Read/AVI_Write with recv/send•FinalizationThursday, March 10, 2011Streaming -- Pause•Video Pause•Skip video frame display•Skip video frame decoding to save computation•Audio Pause•Skip audio chunk play•Keep audio chunk as small as possible (<100ms)Thursday, March 10, 2011A/V Synchronization•Timing Related Func•SDL_GetTicks() : millisecond accruacy•SDL_Delay(): millisecond sleep•TimestampsA1 A2 A3V1 V2 V3 V4 V5first_tick = SDL_GetTicks()PTS = SDL_GetTicks() - first_tickPTS = (counter - 1) * sample_timeRecorderThursday, March 10, 2011A/V Synchronization•Use Timestamp to Sync•Play audio sequentially as the reference•Check the time when a video frame is to be displayed•Sleep or discard video frames if necessaryA1 A2 A3V3Playerfirst_tick = SDL_GetTicks()curr_tick = SDL_GetTicks() - first_tickif curr_tick > PTS, skip frame;else, SDL_Delay(PTS-curr_tick)Thursday, March 10, 2011Bandwidth•Calculate the user-level bandwidth•Count only the bytes actually sent/received •Neglect the TCP/UDP/IP packet header, re-transmission, etc...•Print the stats (bps) every second•Counter and Timer•Byte counter•Timer: timer_create()•Time check when recording/comparing PTSThursday, March 10, 2011Latency•Latency = Tcaptur+Tencode +Tnetwork+Tbuffer+Tdecode+Tdisplay•Asynchronous Timing•SDL_GetTicks() doesn’t work•Clock Synchronization•Get δCapturingEncodingSendRecorderRecvDecodingDisplayPlayerLatency =t2 - (t1 + δ) t1=gettimeofday()t2=gettimeofday()Thursday, March 10, 2011Clock SynchronizationPeer 1 Peer 2RequestResponset1t2t4t3•Approximation•rtt = (t4 - t1) - (t3 - t2)•δ = t2 - (t1 + rtt/2)•Check NTP and other clock sync algorithms for more accurate solutionsThursday, March 10, 2011Big PictureRecorderVideo CaptureJPEG EncodeAudio CaptureA/V MUXWAVvideo windowJPG+WAVPlayerJPEG DecodeA/V DEMUXAudio PlayWAVJPG+WAVJPGYUV/RGBYUYVJPGSync ControlMonitor and ControlMonitor and Controlstats printNetwork InterfaceNetwork Interfacenetwork streamingThursday, March 10, 2011System Framework•Multi-threading -- Strongly Recommended!•Control thread•networking thread, or two network threads (one for audio, one for video)•Audio recording/playing thread•Video recording/playing thread•Use buffersThursday, March 10, 2011Optional Features•Name Server•Register to the name server with both user name and IP/port•Query the name server with the user name for the corresponding IP/port•Remote Control•Print interaction latency•Add tags to the video frameThursday, March 10, 2011Optional Features•Bandwidth Control•BW Control should not affect the video playback by recorder•In order to get full points, you should be aggressive to use as much bandwidth as possible (within limit) to provide the best possible A/V quality. •RTP•http://www.pernau.at/kd/voip/bookmarks-rtp-stacks.htmlThursday, March 10, 2011Comments•Look at submission instructions (10 points)•Write more in your documentation•Explain your negotiation protocol•Describe your program framework•You lose up to 20 points if you write only one sentence per feature•Don’t print fake number!Thursday, March 10, 2011Comments•Audio Settings Thursday, March 10, 2011Comments•Audio Settings Thursday, March 10, 2011Questions?Thursday, March 10,
View Full Document
Unlocking...