Multimedia Week 4 LBSC 690 Information Technology Agenda Questions XML review Images Audio Streaming SMILe What s a Document Content Structure Appearance Behavior History of Structured Documents Early standards were typesetting languages NROFF TeX LaTeX SGML HTML was developed for the Web Too specialized for other uses Specialized standards met other needs Change tracking in Word annotating manuscripts XML seeks to unify these threads One standard format for printing viewing processing Goals of XML Metalanguage A toolkit for design markup languages Unambiguous markup Clear span of tags Separate markup from presentation Style info stylesheet so easy to change Be simple A Family of Standards Definition DTD Names known types of entities with labels Defines part whole and is a relationships Markup XML Tags regions of text with labels Markup XLink Defines hypertext and other link relationships Presentation XSL Specifies how each type of entity should be rendered Some XML Applications Text Encoding Initiative For adding annotation to historical manuscripts http www tei c org Encoded Archival Description To enhance automated processing of finding aids http www loc gov ead Metadata Encoding and Transmission Standard Bundles descriptive and administrative metadata http www loc gov standards mets The Gullibility of Human Senses Three simple tricks for producing Images Video Audio But how do you move the bits around fast enough Seurat Georges A Sunday Afternoon on the Island of La Grande Jatte Visual Perception Closely spaced dots appear solid But irregularities in diagonal lines can stand out Any color can be produced from just three Red Blue and Green additive primary colors High frame rates produce apparent motion Smooth motion requires about 24 frames sec Visual acuity varies markedly across features Discontinuities easily seen absolutes less crucial Basic Image Coding Raster of picture elements pixels Each pixel has a color Binary black white 1 bit Grayscale 8 bits Color 3 colors 8 bits each Red green blue Screen A 1024x768 image requires 2 4 MB So a picture is worth 400 000 words Monitor Characteristics Technology CRT Flat panel Size 15 17 19 21 inch Measured diagonally For CRT key figure is viewable area Resolution 640x480 800x600 1024x768 1280x1024 pixels Layout three dot lines Dot pitch 0 26 0 28 Refresh rate 60 72 80 Hz Some Questions How many images can a 64 MB flash card store But mine holds 120 How How long will it take to send an image at 64kb s But my Web page loads faster than that How But in reality images don t have these problems How do we get around these problems Compression Goal reduce redundancy Send the same information using fewer bits Originally developed for fax transmission Send high quality documents in short calls Two basic strategies Lossless can reconstruct exactly Lossy can t reconstruct but looks the same Palette Selection Opportunity No picture uses all 16 million colors Human eye does not see small differences Approach Select a palette of 256 colors Indicate which palette entry to use for each pixel Look up each color in the palette Run Length Encoding Opportunity Large regions of a single color are common Approach Record of consecutive pixels for each color An example of lossless encoding GIF Palette selection then lossless compression Opportunity Common colors are sent more often Approach Use fewer bits to represent common colors 1 Blue 75 01 White 20 001 Red 5 75x1 75 20x2 40 5x3 15 130 75x2 150 20x2 40 5x2 10 200 JPEG Opportunity Eye sees sharp lines better than subtle shading Approach Retain detail only for the most important parts Accomplished with Discrete Cosine Transform Allows user selectable fidelity Results Typical compression 20 1 Variable Compression in JPEG 37 kB 20 4 kB 95 Discussion Point JPEG vs GIF in Web images Which format should I use for images in my Web pages Color photos Scanned black white text Line drawings Hands On Exercise Convert Between Formats Download and save two images http www umiacs umd edu daqingd image1 jpg http www umiacs umd edu daqingd image2 gif Use Microsoft Paint to convert each to the other format and compare quality and the size Why the difference Discussion Point When is Lossless Compression Important For images For text For sound For video Basic Video Coding Display a sequence of images Fast enough for smooth motion and no flicker NTSC Video 60 interlaced half frames sec 512x486 HDTV 30 progressive full frames sec 1280x720 Video Compression Opportunity One frame looks very much like the next Approach Record only the pixels that change Standards MPEG 1 Web video file download MPEG 2 HDTV and DVD MPEG 4 Web video streaming Basic Audio Coding Sample at twice the highest frequency One or two bytes per sample Speech 0 4 kHz requires 8 kB s Standard telephone channel 1 byte samples Music 0 22kHz requires 88 kB s Standard for CD quality audio 2 byte samples Speech Compression Opportunity Human voices vary in predictable ways Approach Predict what s next then send only any corrections Standards Real audio can code speech in 6 5 kb sec Demo at http www data compression com speech html Scroll down to near the bottom How do MP3s work Opportunity The human ear cannot hear all frequencies at once all the time Approach Don t represent things that the human ear cannot hear Human Hearing Response Experiment Put a person in a quiet room Raise level of 1kHz tone until just barely audible Vary the frequency and plot the results Frequency Masking Experiment Play 1kHz tone masking tone at fixed level 60db Play test tone at a different level and raise level until just distinguishable Vary the frequency of the test tone and plot the threshold when it becomes audible Temporal Masking If we hear a loud sound then it stops it takes a while until we can hear a soft tone at about the same frequency Putting it all together Psychoacoustic compression Eliminate sounds below threshold of hearing Eliminate sounds that are frequency masked Eliminate sounds that are temporally masked Eliminate stereo information for low frequencies Transmission Download Transfer the whole file then start replay Can be very slow for large files Streaming Play the file as it is received Also suitable for live broadcasts Requires a sufficiently fast connection The Last Mile Traditional modems 56 kb sec modems really move 3 kB sec Digital Subscriber Lines 384 kb sec downloads 38 kB sec 128 kb sec uploads 12 kB sec Cable modems 10 Mb sec downloads 1 MB sec 256