Physically Based SoundOverviewMotivationSound Production TodayPBS: Sound Production in NatureMain Aims of PBSChallengesTwo ApproachesMain ideasSlide 10Simulation RequirementsSystem StructureVibration ModellingSound Propagation ModellingSlide 15Surface Vibrations and SoundPutting it all togetherPropagation DelayResults: CapabilitiesDemoResults: AccuracyResults: SpeedSlide 23FeaturesSynthesis MethodVibrationEmissionThe Modal Synthesis ModelExample: A 1D stringSlide 30Force ModelingImpact ForcesSliding/ScrapingSlide 34RollingRolling: Smooth SurfacesRolling: Contact EvolutionSlide 38Slide 39Dynamic ForcesResultsSlide 42DiscussionSlide 44Slide 45ConclusionReferencesAcknowledgementsPhysically Based SoundCOMP259 Nikunj RaghuvanshiOverviewBackgroundFEM SimulationModal Synthesis (FoleyAutomatic)Comparison/ConclusionsMotivationSounds could in-principle be produced automatically, just like graphics: Sound RenderingSound Rendering has not received much research effortMain Goal: Automatic generation of non-music, non-dialogue soundSound Production TodayMovies: Foley Artistshttp://www.marblehead.net/foley/index.htmlGames: Anyone noticed the huge sound directory in Unreal Tournament?PBS: Sound Production in NatureCollisions/Other interactions lead to surface vibrationsVibrations create pressure waves in airPressure waves sensed by earSurface Vibration Pressure Wave EarVibration Propagation PerceptionMain Aims of PBSPhysics simulator gives contact/collision informationAssign material properties for sound, Wood, concrete, metal etc.Sound simulator generates sound using this data (in real time?)ChallengesSound must be produced at a minimum of ~44,000 HzExtremely High Temporal Resolution (timesteps in the range of 10-6-10-8 s)Stiffness of underlying systems (eg. Metallic sounds. K/m~=108)Stability may require even smaller timestepsTwo ApproachesFEM deformable simulationO'Brien, J. F. et. al., “Synthesizing Sounds from Physically Based Motion.” SIGGRAPH 2001.FoleyAutomatic (Modal Synthesis)Kees van den Doel et. Al., “FoleyAutomatic: Physically-based Sound Effects for Interactive Simulation and Animation.” SIGGRAPH 2001.Main ideasDeformable Simulation (arguably) much more “physically based”Foley Automatic: Additive SynthesisComponent SinusoidsSound SignalOverviewBackgroundFEM SimulationModal Synthesis (FoleyAutomatic)Comparison/ConclusionsSimulation RequirementsTemporal ResolutionSimulate Vibration as well as PropagationVibration Modeling: Deformable Model for ObjectsPropagation Modeling: Explicit Surface RepresentationPhysical/Perceptual RealismSystem StructureVibration ModellingFEM with Tetrahedral ElementsLinear Basis Functions, green’s strainExplicit Time IntegrationTypically #nodes = 500, #elements = 1500, dt = 10-6-10-7 sSound Propagation ModellingFluid Dynamic FEM simulation of surrounding air? Very expensive. Instead…Employ Huygen’s Principle: Pressure Wave may be seen as sum of pressure waveletsReceiverReceiverPressure WavePressure “Wavelets”nˆvdsnvzpˆmsPacz /415 Acoustic Impedance of AirSurface Vibrations and SoundPressure contribution of a patch,VelocityDensity of AirSound Propagation Speed in AirUnit NormalSurface Vibrations and SoundApproximate differential elements with surface trianglesApply band pass filters:Low pass: windowed sinc filterHigh pass: DC blocking filterResult: Pressure known for all surface trianglesPutting it all together)cos(~)(rxaptsrxPressure/Signal at ReceiverFiltered Average PressureArea of TriangleVisibility TermApproximation of Beam PatternDistance FalloffnˆReceiverrVibrationxˆPropagation DelayAccumulation BuffercdDelay Receiverd1d2Sourcet=0t1= d1/ct2= d2/c12Receiver Distance from SourceSound Propagation SpeedResults: CapabilitiesGeneral modelsGenerated sounds are accurateStereo SoundDoppler’s EffectDemoResults: AccuracyResults: SpeedScene TimeStep(s) Nodes/Elems Time/Audio TimeBowl 10-6 387/1081 91.3/4.01 minsClamped Bar 10-7 125/265 240.4/1.26 minsVibraphone 10-7 539/1484 1309.7/5.31 mins (~1 day)Timings on a 350MHz SGI Origin MIPS R12K processorOverviewBackgroundFEM SimulationModal Synthesis (FoleyAutomatic)Comparison/ConclusionsFeaturesModal resonance model of solidsLocation dependent soundsImpact, slide, roll excitation modelsReal-time, low latencyEasy integration with simulation/animationPracticalDo not model propagation of sound from source to receiverSynthesis MethodForceForceVibrationVibrationEmissionEmissionPropagationPropagationListenerListenerSpeakersSpeakersSound SamplesSound SamplesUserVibration),(),(]1),([222txFtxutcxxgiiiiSurface u(x,t) of body responds to external contact force F(x,t)u(x,t)F(x,t)Strain Functional Speed of SoundUnder suitable boundary conditions, the solution to the PDE is a sum of sinusoidsEmissionSound pressure s(t) linear functional L of surface vibration u(x,t))],([)( txuLtsiu(x,t)Ls(t)nvzpiiˆ~ Note that propagation is not modeled in aboveThe Modal Synthesis Modelu(x,t)F(p,t)Ls(t)Impulse response/modal model“The response u(x,t) of an arbitrary solid object to an external force can be described as a weighted sum of damped sinusoids”Since L is linear, it implies at s(t) must be a sum of damped sinusoids tooExample: A 1D string 1st Mode 2nd Mode Frequency = f0…Higher modes Frequency = f1= 2*f0 Frequency = fk= k*f0)2sin(000tfeatd)2sin(111tfeatd)2sin( tfeaktdkkMain Idea: Sum contributions of all the modesThe point of impact decides the proportions in which the modes are to be mixed: ak. Therefore, ak is a function of p, the point of impactThe frequencies and damping parameters are a property of the object, and independent of how the object is hit+ +...+a0a1akThe Modal Synthesis Modelu(x,t)F(p,t)Ls(t))2sin()()(1tfepatsktdNkkkImpulse response,modal modelParameters measured experimentallyKth mode: Gain Factor Point Damping Vibration of impact Term FrequencyForce ModelingImpactSlidingRollingWavetableStochasticAt runtime: Find gain parameters given the location, strength and kind of force. Synthesize sound from previous equation.Impact Forces•Duration: hardness (T)•Magnitude: energy transfer (w)•Multiple