Acoustics of SpeechGoal 1: Distinguishing One Phoneme from Another, AutomaticallyGoal 2: Determining How things are said is sometimes critical to understandingToday and Next ClassSound ProductionHow ‘Loud’ are Common Sounds – How Much Pressure Generated?Voiced Sounds are Typically PeriodicSlide 8Slide 9Complex Periodic Waves2 Sine Waves  1 Complex periodic wave4 Sine Waves 1 Complex periodic wavePower Spectra and SpectrogramsExamples and TermsAperiodic WaveformsSlide 16Speech Waveforms in ParticularHow do we capture speech for analysis?SamplingSampling/storage tradeoffSampling ErrorsQuantizationSlide 23FilteringEstimating pitchSlide 26Slide 27Slide 28Slide 29Next Class01/14/19 1Acoustics of SpeechJulia HirschbergCS 470601/14/19 2Goal 1: Distinguishing One Phoneme from Another, Automatically•ASR: Did the caller say ‘I want to fly to Newark’ or ‘I want to fly to New York’?•Forensic Linguistics: Did the accused say ‘Kill him’ or ‘Bill him’?•What evidence is there in the speech signal?–How accurately and reliably can we extract it?01/14/19 3Goal 2: Determining How things are said is sometimes critical to understanding•Intonation–Forensic Linguistics: ‘Kill him!’ or ‘Kill him?’–TTS: ‘Are you leaving tomorrow./?’–What information do we need to extract from/generate in the speech signal? –What tools do we have to do this?01/14/19 4Today and Next Class•How do we define cues to segments and intonation?–Fundamental frequency (pitch)–Amplitude/energy (loudness)–Spectral features–Timing (pauses, rate)–Voice Quality•How do we extract them?–Praat–Wavesurfer–Xwaves…01/14/19 5Sound Production•Pressure fluctuations in the air caused by a musical instrument, a car horn, a voice–Sound waves propagate thru e.g. air (marbles, stone-in-lake)–Cause eardrum (tympanum) to vibrate–Auditory system translates into neural impulses–Brain interprets as sound–Plot sounds as change in air pressure over time•From a speech-centric point of view, sound not produced by the human voice is noise–Ratio of speech-generated sound to other simultaneous sound: Signal-to-Noise ratio01/14/19 6How ‘Loud’ are Common Sounds – How Much Pressure Generated?Event Pressure (Pa) DbAbsolute 20 0Whisper 200 20Quiet office 2K 40Conversation 20K 60Bus 200K 80Subway 2M 100Thunder 20M 120*DAMAGE* 200M 14001/14/19 7Voiced Sounds are Typically Periodic•Simple Periodic Waves (sine waves) defined by–Frequency: how often does pattern repeat per time unit •Cycle: one repetition•Period: duration of cycle•Frequency=# cycles per time unit, e.g. sec.–Frequency in Hz = cycles per second or 1/period–E.g. 400Hz pitch = 1/.0025 (1 cycle has a period of .0025; 400 cycles complete in 1 sec)•Zero crossing: where the waveform crosses the x-axis01/14/19 8–Amplitude: peak deviation of pressure from normal atmospheric pressure –Phase: timing of waveform relative to a reference point01/14/19 901/14/19 10Complex Periodic Waves•Cyclic but composed of multiple sine waves•Fundamental frequency (F0): rate at which largest pattern repeats (also GCD of component frequencies) + harmonics•Any complex waveform can be analyzed into its component sine waves with their frequencies, amplitudes, and phases (Fourier’s theorem)01/14/19 112 Sine Waves  1 Complex periodic wave01/14/19 124 Sine Waves 1 Complex periodic wavePower Spectra and Spectrograms•Frequency components of a complex waveform represented in the power spectrum–Plots frequency and amplitude of each component sine wave•Adding temporal dimension  spectrogram•Obtained via Fast Fourier Transform (FFT), Linear Predicative Coding (LPC),…–Useful for analysis, coding and synthesisExamples and Terms•Vowels.wav, speechbeach1.wav, speechbeach2.wav•Spectral slice: plots amplitude at each frequency•Spectrograms: plots changes in amplitude and frequency over time•Harmonics: components of a complex waveform that are multiples of the fundamental frequency (F0)•Formants: frequency bands that are most amplified by the vocal tract01/14/19 15Aperiodic Waveforms•Waveforms with random or non-repeating patterns–Random aperiodic waveforms: white noise•Flat spectrum: equal amplitude for all frequency components–Transients: sudden bursts of pressure (clicks, pops, lip smacks, door slams)•Flat spectrum with single impulse–Voiceless consonants01/14/19 17Speech Waveforms in Particular•Lungs plus vocal fold vibration filtered by the resonances of the vocal tract produce complex periodic waveforms–Pitch range, mean, max: cycles per sec of lowest frequency component of signal = fundamental frequency (F0)–Loudness: •RMS amplitude: •Intensity: in Db, where P0 is auditory threshold pressureNixixNP120101log10NiixN12101/14/19 18How do we capture speech for analysis?•Recording conditions–A quiet office, a sound booth, an anechoic chamber•Microphones convert sounds into electrical current: oscillations of air pressure become oscillations of voltage in an electric circuit–Analog devices (e.g. tape recorders) store these as a continuous signal–Digital devices (e.g. computers,DAT) first convert continuous signals into discrete signals (digitizing)01/14/19 19Sampling•Sampling rate: how often do we need to sample?–At least 2 samples per cycle to capture periodicity of a waveform component at a given frequency •100 Hz waveform needs 200 samples per sec•Nyquist frequency: highest-frequency component captured with a given sampling rate (half the sampling rate) – e.g. 8K sampling rate (telephone speech) captures frequencies up to 4K01/14/19 20Sampling/storage tradeoff•Human hearing: ~20K top frequency –Do we really need to store 40K samples per second of speech?•Telephone speech: 300-4K Hz (8K sampling)–But some speech sounds (e.g. fricatives, stops) have energy above 4K…–Peter/teeter/Dieter•44k (CD quality audio) vs.16-22K (usually good enough to study pitch, amplitude, duration, …)•Golden Ears…01/14/19 21Sampling Errors•Aliasing: –Signal’s frequency higher than the Nyquist frequency–Solutions:•Increase the sampling rate•Filter out frequencies above half the sampling rate (anti-aliasing filter)01/14/19 22Quantization•Measuring the amplitude at sampling points: what resolution to choose?–Integer representation–8, 12 or 16 bits per sample•Noise due to quantization steps avoided by higher resolution --