Interactive Video Game Group Report An ECE532 Digital Hardware project of 2005 Leo Hwang 990832060 Timothy Li 990964505 March 28 2005 Table of Contents 1 Overview 3 1 1 Goals 3 1 2 System Block Diagram 3 1 3 Brief Description of IP 4 1 4 Brief Description of Software 4 1 5 Clock Domains 4 2 Project Outcome 6 2 1 Review of Original Plan 6 2 2 Results 6 2 3 Compromises 6 2 3 1 Skin Detection Flag Detection 6 2 3 2 Saturation of YCbCr RGB Conversion 6 2 3 3 Lines in Video Output 7 2 4 Suggestions for the Future 7 3 Description of Hardware Blocks 8 3 1 External Memory Controller 8 3 2 OPB GPIO 8 3 3 Video Input Tvin2 8 3 3 1 tvin2 v 8 3 3 2 Video Input Testing 9 3 4 Video Output bm mode svga ctrl 9 3 4 1 Video Output Testing 9 3 5 ZBT Bank Switcher zbt mux 9 3 5 1 Overview 9 3 5 2 Diagram 10 3 5 3 Controlling Buffer Rotation 11 3 5 4 Proper Cycling Order of Ports 11 3 5 5 ZBT Bank Switcher Testing 12 3 6 System mhs and System ucf 12 4 Description of Software 13 4 1 Text Drawing 13 4 1 1 Text Drawing Testing 13 4 2 User Flag Detection 13 4 2 1 User Flag Detection Testing 13 4 3 Video Game 14 4 3 1 Video Game Testing 14 5 Description of Design Tree 15 5 1 Documentation 15 5 2 NTSC in VGA out triple buf 15 5 3 Reference Designs 15 5 4 Snapshots 15 6 References 16 6 1 Module References 16 6 2 Multimedia Board Chip References 16 1 Overview 1 1 Goals The goal of our project was to create a video game where the user interacts by making gestures into a camera Video game elements are superimposed onto the person s outline and then displayed on a monitor 1 2 System Block Diagram Multimedia Board Microblaze Processor Virtex II FPGA Video Game User Detection Text Drawing OPB Bus OPB GPIO Xilinx IP Video Output custom VGA Interface DAC External Memory Controller Xilinx IP ZBT Bank Switcher custom ZBT RAM Bank0 ZBT RAM Bank1 ZBT RAM Bank2 Monitor Video Input custom NTSC Interface ADC Camera Figure 1 Block diagram of project shaded blocks denotes those we created or modified 3 1 3 Brief Description of IP IP Name What it does External Memory Controller OPB GPIO Video Input Video Output ZBT Bank Switcher a k a ZBT mux 1 4 Brief Description of Software Software Block Text Drawing User Detection Video Game 1 5 Where it came from Translates memory accesses on OPB bus into Xilinx IP with memory read write commands minimal changes A set of software addressable input output Xilinx IP registers that can be connected to other IP Reads video from a camera into a ZBT Custom memory bank Writes data from a ZBT bank to the monitor Xilinx example modified to work in XPS Connects three sets of memory controllers to Custom three ZBT memory banks and rotates the connections upon a particular signal What it does Converts a single character number into a bitmapped representation comprised of small squares Finds the user s flag by partition the screen into 8x6 regions and determining which region is most red Blocks fall vertically down the screen and the user must catch them with the flag before they reach the bottom Clock Domains This project due to its video input and output components necessitated the use of different clock domains as shown in Figure 2 Thankfully the domains were naturally self containing and they only interacted with each other indirectly through the rotation of memory banks more on this in section 3 5 Each ZBT bank is driven by the clock that its controlling module is running at There may be an abrupt moment when switching the banks where it is suddenly connected to another clock However switching the banks is only allowed during the vertical blanking interval of the video output and when the software has finished its drawing routines Hence this combination gives a safe period where no data is being transferred to switch the memory banks across modules with their respective clock domains 4 VideoIn clk 27MHz 27 MHz not aligned with System clk System 27Mhz clk DCM 4 3 DCM 2 54 MHz 36 MHz req d VGA timing speed up the uB Tvin2 Microblaze Bm mode svga ctrl ZBT controller OPB EMC Built in ZBT ctrl ZBT Mux ZBT Bank 0 bank1 bank2 ZBT Bank 1 bank2 bank0 ZBT Bank 2 bank0 bank1 Figure 2 Clock Domains found in project 5 2 Project Outcome 2 1 Review of Original Plan The original proposal for the project stated that we would create an interactive video game by using video input to superimpose the user right into the game Of course after the game elements were drawn the combined display needs to be output Tracking the user to allow for the interaction would have used some sort of motion or gesture detection 2 2 Results Overall the outcome of the project was very close to the original plan The final product was able to obtain user input through a camera draw the game and output to a monitor However small compromises were made in three areas of the project These compromises are non detrimental they slightly affect the quality of the result The decision to not pursue their resolution stems from the short timeline for the project 1 Instead of detecting the user s gestures we detect a red object which the user holds 2 Despite getting recognizable images from the video input colours are not being decoded sharply or correctly 3 The quality of the picture output is not as clean as we would like 2 3 Compromises 2 3 1 Skin Detection Flag Detection To detect the user s movements we would need a skin detection algorithm We decided to use something simpler and guaranteed to work a red flag detection algorithm 2 3 2 Saturation of YCbCr RGB Conversion The YCbCr to RGB conversion was producing many false coloured pixels We did not fix the problem but focused on completing the remaining pieces of the project 6 2 3 3 Lines in Video Output After adding the Video Input block to the 3xZBT mux we observed vertical lines in the display This was assumed to be a timing issue The vertical lines did not impede the user detection algorithm so fixing this problem was put on low priority 2 4 Suggestions for the Future Obviously the first place for improvement would be to the aforementioned compromises Any or all of skin detection fixing the YCbCr RGB conversion or cleaning up the video output are worthwhile projects Skin detection would require an implementation of various hue based algorithms which can be found on the internet The latter two would require a deep understanding of NTSC color spaces analog decoder or VGA vga dac workings above what we have learned in the course of the project …