i A Hand Controlled Digital Audio Synthesizer 6.111 Final Report December 13, 2006 Behram Mistree Alexander Sanchez We built a multimedia digital synthesizer. Using subtractive synthesis, we were able to produce notes that sounded similar to a violin and a guitar. The synthesizer is controlled by a computer keyboard interface and video monitoring interface. The video monitoring interface tracks hand movements reasonably well using a video camera. The hand movements in turn control the pitch of the note that is playing. All movements will be displayed to the computer monitor as feedback to the user. We also investigated the use of a neural network in synthesizing audio. Unfortunately, we were only able to test our neural network in simulation.ii Table of contents Abstract i List of Figures v List of Tables vi List of Equations vii Acronyms viii 0. Introduction 1 0.1 Overview of Audio Synthesis System (Alexander Sanchez) 1 0.1.2 Subtractive Synthesis (Alexander Sanchez) 1 0.1.2.1 Neural Network Overview 1 0.1.2.2 Forward Propagation 1 0.1.2.3 Backwards Propagation 2 0.2 Overview of Hand Detection (Behram Mistree) 3 1. Module Descriptions 5 1.1 Module Description for Subtractive Synthesis System (Alexander Sanchez) 5 1.1.1 adsr_FSM 5 1.1.2 note_to_frequency2 6 1.1.2.1 Solution to Issue of Sampling ROMs Storing Waveforms 7 1.1.3 sine_generator 7 1.1.4 sawtooth_wave_generator 8 1.1.5 square_wave_generator 8 1.1.6 synth_channel_selector 8 1.1.7 adsrROM 8 1.1.8 guitarROM 9 1.1.9 sineROM 9 1.1.10 squareROM 10 1.1.11 sawtoothROM 10 1.1.12 violinROM 10 1.2 Module Descriptions for Neural Network System (Alexander Sanchez) 10 1.2.1 propagation_fsm 10 1.2.2 neuron_bus 12 1.2.3 Weight Control System 12 1.2.3.1 weight_manager_controller 12 1.2.3.2 weight_manager 14 1.2.3.3 weight_address_bus 15 1.2.3.4 weight_bus 15 1.2.3.5 weight_system_test_fixture 15 1.2.4 Neurons 15 1.2.4.1 neuron 15 1.2.4.2 weight_multiplier 16 1.2.4.3 input_function 17iii 1.2.4.4 sigmoid 17 1.2.4.5 weight_adjuster 18 1.2.4.6 neuron_output 18 1.2.4.7 weight_multiplier_output 19 1.2.4.8 sigmoid_output 20 1.2.4.9 output_error 20 1.2.4.10 weight_adjuster_output 20 1.2.4.11 network_test_fixture 21 1.3 Block Diagrams for Hand Detection, Serial, etc. (Behram Mistree) 22 1.3.1 High Level Block Diagram 22 1.3.2 Keyboard, ROM, Naive Hand Detection Block Diagram 22 1.3.3 Naive Hand Detection Block Diagram 23 1.3.4 Alternate Hand Detection Block Diagram 23 1.3.5 Audio Player Block Diagram 24 1.3.6 Monitor Module Block Diagram 24 1.3.7 Serial Block Diagram 25 1.4 Module Desccriptions for Hand Detection, Serial, etc. (Behram Mistree) 25 1.4.1 hand_detect 25 1.4.2 alternateHandDetect 26 1.4.3 b_monitor 27 1.4.4 note_module 28 1.4.5 bgetHorVert 30 1.4.6 write_all_out 31 1.4.7 reAttemptSerial 32 2. Testing and Debugging 32 2.1 Testing and Debugging Issues for Subtractive Synthesis (Alexander Sanchez) 32 2. 1.1 General Testing Strategy for Subtractive Synthesis 32 2.1.2 Issues Sampling Stored Waveforms 32 2.1.3 Issues Involving Signed Number Calculations 33 2.1.4 Issues Creating ADSR Envelopes 33 2.1.5 Issues with the ADSR FSM 34 2.1.5.1 Issues With Note Duration 34 2.1.5.2 Issues with Sounding Repeated Notes 34 2.2. Testing and Debugging Issues with the Neural Network (Alexander Sanchez) 34 2.2.1 General Testing Strategy for the Neural Network 34 2.2.2 Issues with Scaling Numbers 35 2.2.3 Issues with Signed Number in Case Statements 35 2.2.4 Possible Solution to Issues with Signed Numbers 36 2.2.5 Precision Error Issues 36 2.2.6 Issues with Constraints on Size of Stored Weights 36 2.2.7 Simulation Results of Neural Network 37 2.2.8 Issues with Compiling the Neural Network 39iv 2.3 Testing and Debugging Hand Detection and Serial. (Behram Mistree) 40 2.3.1 Initial Hand Detection Testing and Debugging 40 2.3.2 Alternate Hand Detection Testing and Debugging 40 2.3.3 Serial Module Testing and Debugging 41 3. Conclusion 42 4. References 43 A. Appendix 44 A.1 Test_guitar2.m 44 A.2 test_violin_r2.m 45v List of Figures Figure 0.11: Mathematical Model of a neuron. 2 Figure 0.21: Drawing of a simple multi-layer neural network. 2 Figure 1.1: Block diagram of adsr_FSM module 6 Figure 1.2: ADSR Envelope for the guitar. 9 Figure 1.3: ADSR envelope for a violin 10 Figure 1.4: State Transition Diagram for the propagation_fsm. 11 Figure 1.5: Block diagram of neural network. 11 Figure 1.6: Block diagram of Weight Control System. 13 Figure 1.7: Block diagram of a neuron. 16 Figure 1.8: Block diagram of neuron_output. 19 Figure 1.9: High Level Block Diagram 22 Figure 1.10: Keyboard, ROM, Naive Hand Detection Block Diagram 22 Figure 1.11: Naive Hand Detection Block Diagram 23 Figure 1.12: Alternate Hand
View Full Document