System OverviewSensing the EnvironmentADV7181 Controller DesignVGA FramebufferReacting to the EnvironmentDMARobotHardware-Software IntegrationNios II Software Recognition AlgorithmConclusionFile Listingstv_controller.vhdadv_interface.vhdyuv2rgb.vhdline_buffer.vhdvga_fb.vhdpots.vhdpots.cimg_tracking.cimg_tracking.hPivoting Object Tracking System[CSEE 4840 Project Final Report - May 2009]Damian AncukiewiczApplied Physics and Applied [email protected] RoyComputer Science [email protected] ShenElectrical Engineering [email protected] ShaoComputer Science [email protected] project implements an object recognition system,where a camera tracks the position of an object. The camerais mounted on an iRobot Create two-wheeled robot, whichrotates according to the control signal generated by our ob-ject tracking algorithm. Meanwhile, it displays 320 × 200color video on a VGA display. We use a simple object recog-nition algorithm based on color information of the imagecoming from the camera. In our tests, the system is able trackobjects of single colors such as white, red, orange or blueif there is sufficient contrast between the object and back-ground.1. System OverviewIn Figure 1, we give an overview of all hardware compo-nents in our system, which are all hooked up to the Avalonbus. Our system works as following: a video camera is con-nected to the Altera DE2 and sends NTSC analog signalsto the board. An Analog Devices ADV7181 converts ana-log video signals from a camera to digital signals in YUVformat. The converter has an I2C interface, which allows forthe output format and other parameters to be configured. TheADV7181 decoder controller takes digital video input fromthe ADV each pixel from YUV to 16-bit RGB. A double linebuffer in the FPGA’s block RAM is used used for data trans-fer between the 27 MHz frequency domain of the video con-troller and the 50 MHz frequency domain of the Avalon bus.Each line of the image data is sent to a buffer in the SDRAMusing DMA (direct memory access) controller. The Nios IIthen performs processing on the buffer in SDRAM in orderto find the center of the object we are tracking and to markup the image. Additionally, the Nios II uses a serial interfaceto command the iRobot Create to turn in the appropriate di-rection if necessary. Subsequently, the buffer in SDRAM issent to the VGA controller again by using DMA, which inturn transfers the buffer to SRAM. This buffer in SRAM isused to display the marked-up image on a VGA screen.2. Sensing the Environment2.1 ADV7181 Controller DesignIn figure 2, we describe the design of the ADV7181controller. The ADV7181 works on a 27 MHz clock andoutputs data in the YUV format on an 8-bit parallel bus.The order used for transferring the information is YUYV,where each Y represents the luma (brightness) componentof a pixel, while the U and V components represent chroma(color) and are shared between the two pixels. Using therecommended settings, each line of video takes up 1716clock cycles, or 858 pixels, although the first 276 cyclesconsist of the horizontal blanking interval, in which no videoinformation is sent. In the first two cycles, the HS signal ispulled low. A frame of video consists of 525 such lines, andis output in interlaced form. The first 20 lines consist of thevertical blanking interval, followed by 242 lines of activevideo which represent alternating lines of the frame. This isfollowed by another blanking interval of 21 lines, followedby 242 more lines of active video, representing the other setof alternating lines of the frame. The VS signal is pulled lowfor three lines in both vertical blanking intervals. Thus, theresolution that the ADV7181 outputs is (1716 − 276)/2 ×242 × 2 = 720 × 484 pixels. This is later downscaled to320×200 pixels due to the limitations imposed by the timingof the VGA framebuffer, as elaborated in the next section.The ADV7181 is highly configurable and has an I2Cinterface for that purpose. To configure the ADV7181 tooutput data in the correct format, an I2C controller, writtenin Verilog, was imported from a lab assignment previouslygiven to us. The controller, when started, sends a set ofrecommended settings to the ADV7181.The ADV7181 interface module is clocked at the samespeed as the ADV7181 itself. It reads the ADV7181’s paral-CSEE 4840 Embedded System Design, Spring 2009 1 2009/5/16Figure 1. System designFigure 2. Design of the ADV7181 controllerlel bus on the rising edge of each clock cycle and incrementsits horizontal counter to keep track of the clock count. Everyfour cycles, 32 bits of information, representing two pixels,are output by the interface module. The module also readsthe state of the HS and VS signals. When it detects that HSis pulled low, it resets the horizontal counter and incrementsthe vertical counter. When it detects that VS is pulled low, itresets the vertical counter and changes the state of its fieldsignal. Thus, at all times the decoder interface knows thehorizontal position, line and field of the raster.The YUV to RGB conversion module takes as input the32 bits in YUYV format representing two pixels from thedecoder interface and outputs a single 16-bit pixel in RGBformat. The first 5 bits represent the green component, themiddle 6 bits are the blue component, and the last 5 bitsare the green component. Since two pixels are convertedinto one, this effectively halves the horizontal resolution.Because there are two Y components for one output pixel,only the first Y component is used. The conversion is doneusing the following formula:RGB=298 0 409298 −100 −208298 516 0Y − 16U − 128V − 128+128128128The R, G and B values are then divided by 256, clipped toa range of between 0 and 255, and packed into the 16-bit5:6:5 format. Because this algorithm can be done entirelyusing integer multiplication and bit shifting, it was easilyimplemented in VHDL.The RGB data then needs to be transferred through theAvalon bus to a buffer on the SDRAM so that the Nios IIcan perform object recognition. This task is made tricky bytwo complications. First of all, the ADV7181 interface andNios system operate in different clock domains: the formerruns off of the same 27 MHz clock as the ADV7181, whilethe latter runs off of the 50 MHz system clock. Secondof all, the SDRAM has latencies that are inherent to itsdesign, and thus it is
View Full Document
Unlocking...