Sound BlocksAn Interactive Environment for Real-Time Music CreationDennis Miaw & Iris Cheung6.11112/13/06AbstractThe purpose of the Sound Blocks is to provide a unique and interactive interface forcreating music in real-time. A user is able to mix, match, and modify sounds by placingand manipulating blocks on an interactive table. There are three types of blocks: source,local-effect, and global-effect. A source block generates sound, such as a drum loop or abass line. Local-effect blocks, when placed near a source block, modify the waveform ofthe source block, producing effects such as reverb and delay. A global effect block controls a property that affect the entire system, such as volume.iiTable of ContentsIntroduction 1Overview 1Image Controller 1Module Descriptions for the Image Controller 2Audio Controller 5Module Descriptions for the Audio Controller 6Implementation of Local Effects 8Memory and Storage of Sound Files 9Testing and Debugging 9Conclusion 9List of FiguresFigure 1: Block Diagram of the Image Controller 2Figure 2: Layout of the Camera View 3Figure 3: Audio Controller 5Figure 4: Audio Controller Modules 6Figure 5: Connections 7Figure 6: Source Module Finite State Machine (FSM) 8Figure 7: Testing Reverb 8Figure 8: Testing Delay 81INTRODUCTIONThe Sound Blocks project consists of a camera which is used to identify colored blocks placed on a table. The location and the colors of the blocks are used to control the playback of music and sounds. Placing blocks on specific regions of the table determines whether or not a sound will be played, or whether or not an effect will be applied to another source. If an effect is applied, the specific location of the block also indicates the magnitude or intensity of the effect. Currently, only red, green, and blue, are being detected, which allows for the capability of triggering three separate sound sources, and three different effects, each of which can be applied to any or all of the sources. The project was implemented with the intention of having four sources and four effects, but time constraints only allowed for the detection of three different colors.The inspiration for the project Sound Blocks came from a project exhibited in the Emerging Technologies SIGGRAPH 2006 convention called the Reactable<http://mtg.upf.edu/reactable/>. Like the Reactable, the Sound Blocks environment allows multiple users to manipulate an intuitive and free form music generation system by placing and moving blocks on a table. Such a system would be entertaining and easy to use for children and adults of all ages. The Sound Blocks system could be used as a learning tool for studying sound and music, or a performance tool used by musicians. The objectives of the project are:- To take a video feed from a camera and design the digital circuitry for determining the location of different blocks on a table based on color-recognition.- To play a sound file which corresponds to a particular source block color.- To associate local-effect blocks with source blocks based on relative position to create sound effects such as delay and reverb.- To implement a global volume block used to change the total volume of the output.- To use location as a variable for changing the magnitude of effect applied to a source.OVERVIEWIMAGE CONTROLLER (Miaw)The image/visual portion of the project deals with receiving and processing the data coming from the camera input. The camera data is used to determine the location of red, green, and blue colored blocks on the table surface, and the positions and colors are used to determine which sound sources should be played back, and which effects should be applied to the sources. The camera data is also sent through image processing modules provided by the MIT 6.111 course to display the camera image onto the computer monitor. Ultimately, the image portion of the project outputs two values: connections, which indicates the sound sources that are present on the table and the effects that should be applied to each source, and magnitude, which indicates the magnitude or intensity of each effect. These two values are used by the Audio Controller portion of the project to2actually produce and output the correct sound. See Figure 1 for a block diagram of the image portion of the project.Figure 1: Block Diagram of the Image ControllerBlock diagram of the image processing portion of the project, showing the modules and their connections.MODULE DESCRIPTIONS FOR THE IMAGE CONTROLLER:Camera Reader Module (Miaw):The Camera Reader receives the data for hsync, vsync, field, YCrCb, and data_valid from the NTSC_decode module. The Camera Reader module performs two functions: it converts the incoming YCrCb data to RGB, and also generates an hcount and vcount for the incoming camera pixels. The color conversion requires a three-stage pipeline, and follows the standard equations for YCrCrb to RGB conversion. The hcount and vcount is generated from the hsync, vsync, and field data. When a data_valid signal is received, hcount increments by one, and when hsync goes high, hcount gets set to zero and vcountincrements by two, since the camera image is interlaced. When vsync goes high, hcountgets set to zero, and vsync gets set to either zero or one, depending on the field value, again because the image is interlaced.Because the color conversion is a three-stage pipeline, the hcount and vcount outputs must also be delayed in order to be in time with the outgoing RGB data. This module also3outputs a ready_data signal and an endframe signal, which correspond to when there is valid data which can be read, and when the end of each frame is reached. These two signals also must be output in time with the rest of the outgoing data signals.Identify Module (Miaw):This module receives the RGB pixel data along with the associated hcount and vcountfrom the Camera Reader module in order to determine the locations of each colored block on the table. The 720x540 pixel camera screen is divided into nine different regions, each of which can contain a red, green, or blue block, with the exception of the region one, which can only contain a red block. Region one controls the overall volume, regions six through nine control the sound sources, and regions two through five control the effects on the sources. A handful of the leftmost and rightmost pixels of the screen are ignored, as these sections tend to be very noisy with respect to color. The useable portion of the hcount
View Full Document
Unlocking...