ECE532 Digital Hardware Group Project Report Audio to MIDI Converter James Shu Hen Chen Sang Joon Lee 991163904 990908354 Table of Contents TOPIC Pages 1 Overview of the Project 3 1 1 Objective 3 1 2 Background 3 1 3 Organization 4 2 Outcome 6 3 Description of IP Blocks 7 3 1 AC 97 Sound Controller 7 3 2 External Memory Controller Xilinx 7 3 3 UART Xilinx 7 3 4 Fast Simplex Link Xilinx 7 3 5 FFT Wrapper 8 3 5 1 Description 8 3 5 2 Theory of Operation 8 3 5 3 Limitation 8 3 5 4 Design Parameters and Signals 9 3 5 5 Finite State Machine Description 11 3 5 6 Simulation 13 3 6 Software Sound Processor 13 3 6 1 Description 13 3 6 2 Theory of Operation 14 3 6 3 Limitations 14 3 6 4 Testing 15 4 Design Tree 16 References 17 Appendix A FFT Wrapper Core 18 Appendix B Simulation Results 25 2 1 Overview of the Project 1 1 Objective The initial objective of this project is to implement an Audio to MIDI ATM converter on the Xilinx Vertex II Multimedia Board The board would sample audio music signals as an input and outputs MIDI sequences corresponding to the music in real time 1 2 Background Musical Instrument Digital Interface MIDI is a standard in transmitting musical audio information in digital format The standard is supported by most musical synthesizers where the musical notes are synthesized and or manipulated An ATM Converter adds MIDI compatibility to non MIDI instruments It converts audio signals produced by conventional instruments into the MIDI standard thus allowing digital manipulation of the musical notes The audio information can be outputted to a MIDI synthesizer Additionally the ATM Converter can record music into compact MIDI data files A good introduction to the MIDI standard can be found at 1 and 2 3 1 3 Organization Microphone AC97 Codec FSL AC97 Controller FSL MicroBlaze Processor FSL FSL FFT Wrapper custom FFT xfft v6 1 Xilinx IP OPB Interface UART Controller Xilinx External Memory Controller Xilinx IP ZBT External Memory Xilinx Virtex II FPGA I O from serial port RS232 Serial I O Xilinx Multimedia Board Figure 1 System Block Diagram AC97 Controller Responsible for capturing audio sample External Memory Controller Required to access the ZBT memory UART Controller Allows system to output MIDI sequences through the RS232 Port FFT Core Performs FFT operations for 1024 points 4 FFT Wrapper Allows the FFT core to interface with the FSL bus MicroBlaze Processor Processes FFT data to determine whether or not a note has played or stopped and sends MIDI sequences to the RS232 port corresponding to these events Fast Simplex Link FSL Allows communication between the MicroBlaze Processor software and the various hardware blocks 5 2 Outcome Because of the various difficulties we encountered with the FFT our system is unable to convert analog audio signals to MIDI sequences However the system is capable of capturing audio samples at 8000 Hz and determining which music note has been played from the hard coded FFT data Upon detecting a note has been played series of bytes will be outputted from the system through the RS232 port representing the start of a note the MIDI channel index of the note and the velocity loudness We tested the rest of the system with a function that generate stub FFT outputs and everything appears to be functioning correctly Since the MIDI interface is not directly compatible to the RS232 interface a serial to MIDI converter must be attached between the RS232 port at the multimedia board and the MIDI synthesizer A comparatively inexpensive solution can be ordered from http www ittymidi com converter box info asp Future Improvements To reproducing the music more accurately it is possible to monitor the time domain audio information to detect any significant change We can then perform an FFT operation using the audio data follows immediately after the change This not only minimizes the variance in the delay between a note is played and the MIDI sequence is sent it also can give a better representation of the velocity loudness of the note Another possible improvement is of course to get the FFT wrapper working Also using a radix 4 or pipelined version of FFT may slightly improve the performance 6 3 Description of IP Blocks 3 1 AC 97 Sound Controller The AC97 Sound Controller IP block was provided by previous year s project The core is responsible for controlling serial data flow between the external AC97 CODEC chip and the FPGA The controller provides the user with capturing sound and playback audio signal through the FSL bus 4 This IP block was used to capture audio signal and stored in the ZBT memory to perform FFT Approximately 6 ms before disabling sound recording the captured sound will contain gibberish data The sound samples within this timeframe would be discarded Please refer to AC97 Controller document from pervious year for more detail 3 2 External Memory Controller Xilinx The external memory core interfaces the FPGA to the external memory The data is received from the AC97 Controller is written and stored in the external memory These data will later be processed with a FFT module Also the software core is stored in the external memory Please refer to ZBT memory data sheet provided by Xilinx 3 3 UART Xilinx This is the Uart peripheral for input output used by the Xilinx Xilkernel It is used to output the MIDI sequences to the RS232 port 3 4 Fast Simplex Link Xilinx The Fast Simplex Link FSL is a uni directional point to point communication channel bus used to perform fast communication between any two design elements on the FPGA when implementing an interface with the FSL bus The Fast Simplex Link FSL Bus v2 00a was implemented in this project To implement real time audio processing capability the FSL bus was used to directly read from the AC97 core as well feed the audio signal from Microblaze to FFT core and retrieve result from FFT core Please refer to Fast Simplex Link FSL Bus v2 00a data sheet provided by Xilinx for detailed information regarding this IP block 7 3 5 FFT Wrapper 3 5 1 Description The Fast Fourier Transform FFT wrapper responsible for communicating controls signals to the FFT core through the FSL buses The wrapper handles data inputs from FSL FIFO and signal synchronization issue between the FFT core and FSL buses The FFT core used is xfft v3 1 provided by Xilinx This FFT core was generated using the LogicCore tool provided by Xilinx Features FFT Architecture option Radix 2 Uses minimum resources Forward complex FFT only Transform