Slide 1OverviewSlide 3What is sound?What do we hear?PsychoacousticsSlide 7Slide 8Slide 9Digital representation of musicCompressionChoosing a representationSlide 13Analyzing speechAuditory Scene AnalysisMusic information retrievalMachine learning for analysisSlide 18Creating music: SynthesisFour approaches to synthesis2. FM Synthesis3. Physical Models4. Cross-synthesisHow can computers be used in making music?Computer as InstrumentQuestions: How can we….Final remarksSlide 28Digital audio and computer musicCOS 116, Spring 2012Guest lecture: Rebecca FiebrinkOverview1. Physics & perception of sound & music2. Representations of music3. Analyzing music with computers4. Creating music with computers1. Sound and musicWhat is sound?Discussion Time“Pressure wave”What do we hear?PitchLoudnessTimbreLocationMeter, rhythm, harmony, melody, structureetc...http://www.youtube.com/watch?v=EvxS_bJ0yOUhttp://www.youtube.com/watch?v=wY1EMwDeaBwhttp://www.youtube.com/watch?v=nIt9QF_5C_wPsychoacousticsPsychoacoustics: relationships between physical phenomenon and our perceptionFrequency: pitch (20-20,000Hz)Amplitude: loudnessTimbre: Identities and strengths of frequencies present+ =Discussion TimeWhat is music?“Organized sound” •Psychoacoustics play an important role•Also dependence upon history, culture, experience•Engages listeners’ psychological mechanisms for expectation/reward2. Representations of sound and musicScore: Digital waveformSpectrogramHow do you represent music?Digital representation of musicCompressionA “better” representation with fewer bitsWhy? Security, transmission, storageHow?Psychoacoustic principlesMP3: MaskingPhysical principles of sound production (uses models of sound source)Choosing a representationRepresentations make compromisesStandard representations are somewhat arbitraryAppropriate choice is task-dependent3. Using technology to analyze sound and musicAnalyzing speechReal-life apps:Customer service phone routingVoice recognition softwareAuditory Scene AnalysisApplications: Archival and retrieval, forensics, AIMusic information retrievalAnalyzing musical dataQuery, recommend, visualize, transcribe, detect plagiarism, follow along scoreSites/apps you can trymidomiThemefinder.comPandora.com (includes “human-powered” algorithms)ShazaamMachine learning for analysis4. Using technology to create music and soundCreating music: SynthesisFour approaches to synthesis1. Additive synthesis1. Figure out proportions of various frequencies2. Synthesize waves and superimpose them3. Modify amplitude using an “envelope”:+ + …=2. FM SynthesisModulate the frequency of one sine oscillator using the output of another oscillator3. Physical Models1. Start with knowledge of physical systems2. Simulate oscillation (Recall Lecture 4)4. Cross-synthesisChoose filter for speech (vowel)Choose source to be another soundHow can computers be used in making music?Synthesizing new soundsProcessing and transforming soundDemo: T-PainAccompanying human performersDemo: RaphaelComposing new musicDemo: CopinAs new musical instrumentsAnd many other ways, too…Computer as InstrumentDemo: SMELT keyboard, motionVideo: ClixDemo: WekinatorVideo: CMMV, BlinkyDemo: Live codingQuestions: How can we….develop new ways to synthesize sound?give a user control over synthesis parameters?make machines interactive in a musical way?augment human capabilities?design new instruments that are easy to play? allow expert musicality?create music that is emotionally and aesthetically compelling?Final remarksDistinctions in this presentation are superficialAnalysis, representation, and creation interactTechnology draws on and contributes to our understanding of the physics and psychophysics of soundComputer music is interdisciplinaryHCI, AI, programming languages, algorithms, systems buildingAlso psychology, music theory, acoustics, signal processing, engineering, physics, performance practice, library science, applied math & statistics, …Technology is constantly complicating and changing the landscape of our musical experiences as creators, participants, listeners, and
View Full Document
Unlocking...