CardCounterC_codelab5histocommunicationCCD_CaptureCardCounter Final Project Report Team Members: Christos Savvopoulos Hugh Gordon Nathan RoganTable of Contents 1. Introduction 2. Project Design i. Design ii. Components iii. Software 3. Implementation Issues i. Hardware & Interfacing ii. Lighting iii. Physical 4. Lessons and Work Done i. Chris ii. Nate iii. Hugh 5. Code i. C code ii. lab5 (top-level entity) iii. histo (histogram counter) iv. communication v. CCD_Capture (edited)1. Introduction We have written a playing card recognition system using the CCD camera provided with the DE2 board. Using camera input and specialized histogramming techniques, our project detects the value and suite of a given playing card. Given 10 minutes of training, our system will work with a large variety of different playing cards. The image data comes from the CCD camera into the DE2 board as a stream of pixels. As each pixel is received, its color is determined using a set of hard coded thresholds. Each time a pixel of a given color arrives, a register corresponding to that color is incremented. Once the entire picture is received, the software can use the collected data to determine how many pixels of each color are in the picture. Each playing card has a unique number of colored pixels on it. The software program that we wrote on top of the NIOS processor uses this data to determine which card has been placed. While this method is not 100% effective, mainly due to noisy data from the CCD camera, using range and averaging techniques has allowed us to achieve very high rates of accuracy in card recognition. It is worth noting that throughout the development process we implemented unsuccessfully various solutions to solve different issues that had arisen. Some of these failed solutions and the reasoning behind there implementation as well as their failure are included in Appendix A.2. Project Design & System Architecture i. Design When we set off to design this system we had two goals in mind: recognizing cards by counting pixels and separating the tasks better suited to software or hardware. Some parts of the design are inherently hardware, such as the CCD Capture logic and the VGA output. Other tasks, such as an algorithm that recognizes cards are clearly better done in software. To count the number of pixels of a certain color, there are multiple approaches. In the end, we decided to implement the system such that as the CCD_Capture unit reads the data, they are counted by multiple histogram components. When a new frame arrives, the counter is stored in a register. On the software side of things, we want to know the number of pixels for each color. The communication between the two worlds happens through the Avalon bus. In addition to that, to make the system more flexible we decided to give the program the power to define the color of each counter on the fly. This is done specifying a minimum and maximum red, green, and blue component. Not only did this save a huge amount of compiling time (since software compiles a lot faster than hardware) but it also made the system a lot more expandable, since different algorithms could try to change the thresholds on the fly. ii. Components CCD_Capture: Reads from the CCD Camera, and outputs the data component of the currently read pixel. Clock speed: 25 MHZ Input: Clock, GPIO_1 Output: data (10 bits), new frame flag, new pixel flag RAW2RGB: The CCD is a 2D Array of either red, green, or blue sensors. This component does some basic image processing (averaging pixels) and gives out the red, green and blue components for each pixel. Input: pixel clock, data (10 bits), X & Y position Output: red, green, blue (10 bits each)Histogram: This histogram logic takes in the stream of RGB data from the camera logic. Each module (there are 8) is configurable from the software to count up pixels that match a given set of tolerances placed on the red, green and blue values. For instance, to match red, one could set the R tolerance to be between 700 and 1023 and the G and B tolerances to be from 0 to 300. The suitability of these values is highly dependent on the environment. Input: min and max threshold (32 bits each) Output: Count (32 bit wide) of how many pixels fit the RGB constraints VGA Controller: The VGA controller takes the data from the camera logic and turns it into a VGA signal so that it can be displayed on a normal computer monitor. Input: r, g, b: 10bits each Output: VGA signalsConfig and Buttons: The buttons were configured as input to control the settings on the camera. We needed a way to adjust the exposure and turn the camera on and off. The buttons on the DE2 board provided a simple solution. NIOS 2e: The standard processor used with the DE2 board. The edition we used had a 50MHZ clock, 524K of onboard SRAM and used the JTAG UART system to communicate with the terminal. Communication: This component, as the name suggests, was responsible for communicating data between the hardware part and the software part of our solution. It uses the Avalon bus to perform 32 bit reads (accum) and writes (min, max) from/to the hardware. Input: 8 x accum (32 bits each) Output: 8 x min, max (32 bits each) iii. Software Aside from counting the pixels of various colors, this is where all the image processing got accomplished. The software program has two modes of operation. The first mode is the learning mode. In this mode, the user is asked to place cards down and type in their values (i.e. king of spades or 2 of clubs). The software then records range of values that are read for number of red and black pixels over a large number of frames. These two ranges (one for red and one for black) are unique for each card. Once the ranges are paired with a given card, that card can be recognized. The second mode of operation is the reorganization mode. In this mode, the software accesses its previously learned data and attempts to match a given card to it. In this case, instead of taking the range of values over a large number of frames, it takes the average of these values and looks to see weather or not this average fits inside each given remembered range. If it does, the corresponding card value is printed out, otherwise, it tries again. Pseudo code is provided below. pseudocode setThresholds(); //Learning phase while(1) { read
View Full Document