Lecture notes of Image Compression and Video Compression series 6 Perceptual Audio Coding Prof Amar Mukherjee Weifeng Sun Topics z Introduction to Image Compression z Transform Coding z Subband Coding Filter Banks z Introduction to Wavelet Transform z Wavelet Image Compression z Perceptual Audio Coding z Video Compression 2 1 Contents z Sound z Physiology of the Ear z Human Sound Perception Psychoacoustics z Auditory Masking z Frequency masking z Temporal masking z z MPEG Audio Compression z Example 3 Motivation z Many applications need digital audio z Storing z z z Streaming z z z ipod games Internet radio Interactive multimedia session like VOIP conferencing However cannot apply entropy coder directly z Very random missing repeated sequence of byte pattern 4 2 Sound z Sound is a continuous signal that is created by compressing and decompressing the air z This changing air pressure causes the eardrum to vibrate 5 Describing Sound z Intensity z z Defined as power per unit area The level of perceived sound by a human is referred to as loudness z z z Pitch z z Measured in terms of phons Defined as the perceive intensity at 1000 Hz by normal human population frequency of sound tone Quality timbre 6 3 Decibels z For most audio specifications sound levels are presented in terms of decibels dB z z z A log scale for intensity changes The softest 1000 Hz tone heard by the normal ear is I0 1 picowatt meter2 With I0 as a reference all other intensities I can be given in terms of dB by 7 Physiology of the Ear z Thousands of microphones z z hair cells in cochlea Automatic gain control z muscles around transmission bones z Directivity z Boost of middle frequencies z z z pinna auditory canal Nonlinear processing z auditory nerve 8 4 The Outer Ear z z For a point source of sound it spreads out according to the inverse square law For a given sound intensity a larger ear captures more of the wave and hence more sound energy 9 The Tympanic Membrane z z The tympanic membrane or eardrum receives vibrations traveling up the auditory canal and transfers them through the tiny ossicles to the oval window the port into the inner ear The eardrum is some fifteen times larger than the oval window giving an amplification of about fifteen compared to the oval window alone 10 5 The Ossicles z z z The three tiniest bones in the body form the coupling between the vibration of the eardrum and the forces exerted on the oval window of the inner ear Serves as an amplifier with a factor of about three under optimum conditions Can be adjusted by muscle action to actually attenuate the sound signal for protection against loud sounds 11 The Inner Ear Two Organs z Semicircular canals z z body s balance organ Cochlea z z body s microphone converting sound pressure impulses from the outer ear into electrical impulses which are passed on to the brain via the auditory nerve 12 6 The Cochlea z z z Snail shell like structure Three fluid filled sections The organ of Corti is the sensor of pressure variations 13 Place Theory z z z High frequency sounds selectively vibrate the basilar membrane of the inner ear near the entrance port the oval window Lower frequencies travel further along the membrane before causing appreciable excitation of the membrane The basic pitch determining mechanism is based on the location along the membrane where the hair cells are stimulated 14 7 The Auditory Nerve z Taking electrical impulses from the cochlea and the semicircular canals the auditory nerve makes connections with both auditory areas of the brain 15 Auditory Area of Brain z Binaural z relay When the auditory nerve from one ear takes information to the brain that information is directly sent to both the processing areas on both sides of the brain 16 8 Perceptual Audio Compression z The basis of the Perceptual Codecs is Psychoacoustic Masking z An audio file will contains sounds that are not heard by us even though these sounds lie within the human audible range z Masking Techniques z Frequency Concurrent Masking z Temporal Masking 17 Dynamic Range of Hearing z The practical dynamic range could be said to be from the threshold of hearing to the threshold of pain Threshold of Hearing I0 0 decibels z Threshold of Pain 1013I0 10 000 000 000 000 I0 130 decibels This remarkable dynamic range is enhanced by an effective amplification structure which extends its low end and by a protective mechanism which extends the high end 18 9 Frequency Response of the Ear z The ear has a non flat frequency response Tones played at the same volume with different frequencies can sound like they are being played at different volume levels z In order to maintain the same loudness over the hearing spectrum the intensity level must vary z 19 Equal loudness Curve z Audible frequency range z z 20 to 20 000 Hz Normal voice range z z z 500 Hz to 2 kHz Low frequencies are vowels and bass High frequencies are consonants 20 10 Human Hearing Sensitivity z Experiment z z z z Put a person in a quiet room Raise level of 1 kHz tone until just barely audible Vary the frequency Plot 21 Frequency Concurrent Masking z z Experiment Play 1 kHz tone masking tone at fixed level 60 dB Play test tone at a different level e g 1 1 kHz and raise level until just distinguishable Vary the frequency of the test tone and plot the threshold when it becomes audible 22 11 Frequency Concurrent Masking z z Frequency masking effect dependent on frequency As shown people can detect lower frequency test tones closer than higher frequency test tones 23 Frequency Concurrent Masking z z Sub bands are uniform but human hearing sensitivity isn t so psychoacoustic model and MPEG encoder compensates by preserving more data in lower filtered sub bands There are about 25 critical bands 24 12 Temporal Masking z z Experiment Play 1 kHz masking tone at 60 dB plus a test tone at 1 1 kHz at 40 dB Test tone can t be heard it s masked Stop masking tone then stop test tone after a short delay Adjust delay time to the shortest time when test tone can be heard Repeat with different level of the test tone and plot 25 Temporal Masking z Postmasking z z Premasking z z If the volume drops sharply then the human ear takes a few milliseconds to adjust during this period the lower volume sounds are not heard and these can be discarded If the volume rises sharply then the human ear discards the last few milliseconds of the quieter sound and immediately starts processing the louder sound The last few milliseconds of data