Conductor HeroNatalie Cheung, Yuta Kuboyama, Edgar TwiggIntroductionGoal: Conduct an Orchestra based on Hand Movements•Score transferred through a USB Port•User conducts the entire orchestra and smaller sections•Output orchestra music and graphical representation of the orchestra on the screenOverviewUSBInputScore DataScore ManagerConductorMixerenbenbpitch pitchpitchdynamicsarticulationsection_selvideo_outUSB245Mnote_databeat_numbusynote_data(no beat)video_inputenbtempobeatstereo_left,stereo_rightstereo_left, stereo_rightstereo_left, stereo_rightaudio_outreprogram2212107778832x162x162x162x24Compressorsignal_sumresetresetresetZBTnote_dataaddressaddress191934 342x16clk2 clk2clk2clk2clk2clk2clk1ZBT ZBTwrite_enbread_enbVideoDecodervideo_dataInstrument ManagerresetUSBInputScore DataScore ManagerConductorMixerenbenbpitch pitchpitchdynamicsarticulationsection_selvideo_outUSB245Mnote_databeat_numbusynote_data(no beat)video_inputenbtempobeatstereo_left,stereo_rightstereo_left, stereo_rightstereo_left, stereo_rightaudio_outreprogram2212107778832x162x162x162x24Compressorsignal_sumresetresetresetZBTnote_dataaddressaddress191934 342x16clk2 clk2clk2clk2clk2clk2clk1ZBT ZBTwrite_enbread_enbVideoDecodervideo_dataInstrument Managerreset•Score Management- Music data handling- Takes numeric user signals and score data to determine what instruments play whenUSBInputScore DataScore ManagerConductorMixerenbenbpitch pitchpitchdynamicsarticulationsection_selvideo_outUSB245Mnote_databeat_numbusynote_data(no beat)video_inputenbtempobeatstereo_left,stereo_rightstereo_left, stereo_rightstereo_left, stereo_rightaudio_outreprogram2212107778832x162x162x162x24Compressorsignal_sumresetresetresetZBTnote_dataaddressaddress191934 342x16clk2 clk2clk2clk2clk2clk2clk1ZBT ZBTwrite_enbread_enbVideoDecodervideo_dataInstrument Managerreset•Conductor- User Interface- Generates numeric signals from user movement•Score Management- Music data handling- Takes numeric user signals and score data to determine what instruments play whenUSBInputScore DataScore ManagerConductorMixerenbenbpitch pitchpitchdynamicsarticulationsection_selvideo_outUSB245Mnote_databeat_numbusynote_data(no beat)video_inputenbtempobeatstereo_left,stereo_rightstereo_left, stereo_rightstereo_left, stereo_rightaudio_outreprogram2212107778832x162x162x162x24Compressorsignal_sumresetresetresetZBTnote_dataaddressaddress191934 342x16clk2 clk2clk2clk2clk2clk2clk1ZBT ZBTwrite_enbread_enbVideoDecodervideo_dataInstrument Managerreset•Conductor- User Interface- Generates numeric signals from user movement•Score Management- Music data handling- Takes numeric user signals and score data to determine what instruments play when•Sound Synthesis- Given on/off and pitch- Generates actual soundTime24ticksbeatbeatbeat•128 ticks per beat•At 50 bpm a tick lasts for 9.4ms- humans can detect differences of ~20-30ms•Uneven note durations (e.g. triplets) won’t sound choppyNoteData FormatStart BeatStart TickInstrumentPitchEnable0-11 (0-4095)12-18 (0-127)19-25 (0-127)26-32 (0-127)33 (0-1)•Each NoteData takes up one 36 bit wide SRAM location•Two NoteData events per note (on and off) = 72 bits per note•500 Measures in 4/4, average of 15 notes per beat = 2.06MbScore Managementnote_data note_datanote_dataUSBInputZBT Score Dataaddress1934 34address19USB 245Mreprogram2212Score Managerresetresetresetpitch7enablebeattempo10Tick Counterpitch7enable pitch7enablereset7beat_numcurrent_ticknote_data note_datanote_dataUSBInputZBT Score Dataaddress1934 34address19USB 245Mreprogram2212Score Managerresetresetresetpitch7enablebeattempo10Tick Counterpitch7enable pitch7enablereset7beat_numcurrent_tickputs USB data into ZBT SRAMnote_data note_datanote_dataUSBInputZBT Score Dataaddress1934 34address19USB 245Mreprogram2212Score Managerresetresetresetpitch7enablebeattempo10Tick Counterpitch7enable pitch7enablereset7beat_numcurrent_tickputs USB data into ZBT SRAMWhen beat_num > current beatbusy = highoutput new note_datanote_data note_datanote_dataUSBInputZBT Score Dataaddress1934 34address19USB 245Mreprogram2212Score Managerresetresetresetpitch7enablebeattempo10Tick Counterpitch7enable pitch7enablereset7beat_numcurrent_tickputs USB data into ZBT SRAMWhen beat_num > current beatbusy = highoutput new note_dataIf beat switches stateincrement beat_numwhile Score Data is busyoverwrite BRAM w/ new note_dataall data in BRAM now has same beatIf current_tick == tick in BRAMdo the note (turn on/off, set pitch, etc)note_data note_datanote_dataUSBInputZBT Score Dataaddress1934 34address19USB 245Mreprogram2212Score Managerresetresetresetpitch7enablebeattempo10Tick Counterpitch7enable pitch7enablereset7beat_numcurrent_tickputs USB data into ZBT SRAMWhen beat_num > current beatbusy = highoutput new note_dataIf beat switches stateincrement beat_numwhile Score Data is busyoverwrite BRAM w/ new note_dataall data in BRAM now has same beatIf current_tick == tick in BRAMdo the note (turn on/off, set pitch, etc)counts clock cyclescompares to #cyc/tickif a tick has passedincrement tick counterUSB Input (Data Reading Timings)- excerpt from FTDI UM245R USB-Parallel FIFO Development Module DatasheetTick CounterTick Counterreset7tempo10•Counts the # of clock cycles that have elapsed since the last tick•Calculates the # of clock cycles that should elapse before the next tick based on the tempo•Increments the current tick when count = calculatedt empo4beatsmin_128t ickmin_160minsec= t empo_815ticksec1t empo_158sect ick_27,000,000clksec=50,625,000t empoclkt ickConductorConductor ModuleVideo Decoder•Detect LED using thresholds•Reject other colors•Store data into ZBTConductor ModuleZBT•Each frame- ZBT1 collects information from video_data- Conductor module reads from ZBT2•Every frame- ZBT1 <= ZBT2- ZBT2 <= ZBT1Conductor ModuleConductor•Determine if there are 2 LEDs (conduct the whole orchestra)•Determine position of LED•If position of LED remains the same for a certain number of frames (based on a counter), the next time the LED moves to a different position, the beat enable is high•Tempo is based on the velocity of the LEDSound SynthesisUSBInputScore DataScore ManagerConductorMixerenbenbpitch pitchpitchdynamicsarticulationsection_selvideo_outUSB245Mnote_databeat_numbusynote_data(no beat)video_inputenbtempobeatstereo_left,stereo_rightstereo_left, stereo_rightstereo_left,
View Full Document
Unlocking...