This preview shows page 1-2-3-4-5-6-7-8-56-57-58-59-60-61-62-63-114-115-116-117-118-119-120-121 out of 121 pages.
Embedded Image Capture CSEE 4840 Final Report − May 2010 Albert Jimenez Alexander Glass Nektarios Tsoutsos School of Engineering and Applied Science Columbia University {alj2110, amg2229, nt2283}@columbia.edu 1 Overview For our final project, we chose a number of DE2 functions and integrated them. The DE2 supports real time video capture, VGA display, Ethernet transmit and receive, SDRAM memory, SRAM memory, PS2 interface, 7 Segment displays, LEDs, dip switches and many others. Our project leverages all these functions supported by the DE2 to perform hardware based compression of Jpeg images captured via the video input. Our images are displayed in a custom GUI via the VGA display and support two different transfer modes. In one of the transfer modes, images are transmitted board-to-board via UDP. A user initiates a transfer via the keyboard and an image is sent to another board. The image is saved to the SDRAM of the remote board and when the remote user scrolls through the local image list, they can view the transmitted image. In the second transfer mode, images are transmitted board-to-computer via UDP. The computer runs a server program developed with BSD sockets that waits for file transfers from the board. After a new image is received, the server program writes the image to disk and opens the image with an “Eye of Gnome”, an image viewing utility provided with Ubuntu. The project was significantly challenging to complete and we were only able to confirm the correct functionality of all implementations on the final day of the deadline. Some of the hardest challenges we faced included the following: Trying to fit the jpeg encoder and appropriate memory buffers within the constraints of the device; with additional memory elements our jpeg encoding would have been at the full VGA size of 640x480 instead of the scaled down size we were forced to use. Writing to the SRAM frame buffer was problematic because of the synchronization issues involved, such as determining the start of a new frame and contention for SRAM data during reads and writes. The Ethernet component board-to-computer transfer involved several challenges due to packets being discarded by the BSD socket code and operating system related issues to the Ethernet interface with Ubuntu. Despite seeing our packets with Wireshark, (an open source packet capture tool), our host application refused to acknowledge receiving new packets via the recvfrom method.2 Hardware description Here we present a top level view of the system: Figure 1 Top level view of the system 2.1 The Video Capture Hardware The Graphical User Interface The user interface offers a variety of functionality to capture, store, send and view images. All of the functionality is achieved through a ps2 keyboard that communicates with the processor. The interface has two main display modes, full screen and true resolution. It also allows the user to freeze the live image in either display modes. In order to achieve a still image, pixel information is continually stored in an SRAM, which can store one fourth of a full screen image uncompressed. When a stored image is viewed at full screen, the image is interpolated in order to fill the entire screen. In real-resolution mode, stored images are displayed in the exact resolution in which they are stored. The user also has the ability to save multiple images on an SDRAM and view them in either full or real-resolution mode. Once images are saved to the SDRAM, they can be sent remotely to either a remote board or a computer.Keyboard Control Signals Keyboard Button Functionality Spacebar Freeze image Up and down keys Change screen mode Delete Request image deletion Left key Move to next image Right key Move to previous image Enter Save image Right Shift Enter Send image to remote board Left Shift Enter Send compressed image to computer TV to VGA Module The TV to VGA module is responsible for converting the TV signal into a format that can be displayed on a VGA. The module is also responsible for storing a local frame buffer on SRAM and communicating with the NIOS processor in order to send and receive images. The Altera DE2_TV tutorial was used as a starting point to convert and manipulate the incoming TV signal. Figure 2 Block diagram of the tv to vga convertor The initial conversion is achieved by a tv decoder chip (ADV7181), which takes standard tv signal at 50 or 60 hz. This project was tested at 60 hz using the NTSC format. The tv decoder is connected to a I2C AVconfiguration chip which sets up and synchronizes the decoder. The output of the tv decoder is sent to a ITU R565 chip, which is responsible for down-sampling pixels and extracting color information in YCbCr format. This format describes the luminance and chroma of pixels; however in order for this information to be used by the VGA display it must be converted to standard RGB format, which encodes pixel color information in three color channels. The color conversion is done by the YCbCr to RGB module and the result is sent to digital to analog convertor in order to be used directly by the VGA. Proper synchronization is achieved by the VGA timing generator, which uses the original tv synchronization signals and provides the necessary horizontal and vertical sync to the VGA. These synchronization signals were also used in order to keep vertical and horizontal counts that indicate exactly where on the screen a pixel is being displayed. Such counts were also essential in mapping the screen raster onto a consistent location in SRAM and sending the correct horizontal and vertical positions when storing and saving images. Sending From the Frame Buffer to SDRAM Address Number Functionality 0 Send x position 1 Send y position 2 Send request 3 Receive pixel Figure 3 Image exchange protocol from the tv to vga module to the nios When the user presses enter, the image on the screen is first captured on the SRAM and then sent to be stored on the SDRAM. A strict protocol is maintained between the NIOS processor and the TV to VGA module, to ensure that the image is sent properly. First the processor sends a pixel request signal to the TV to VGA module, along with the x and y position. Once it receives the pixel, the processor stores it in SDRAM and repeats the protocol with new x and y
View Full Document
Unlocking...