Picture in Picture Edward Semper Vincent Wu Sue ZhengsThis project is to create a design for a picture in picture system. The system is design to take in two NTSC video sources and display both of them on a VGA screen at the same time. One of the video sources would be displayed in a small window in the foreground while the other would be full size in the background. The choice of which video image is in the foreground and background can be switched by the user. The size and angle of the smaller image can also be adjusted by user input. This design also features and integration mode where an object in the foreground can be integrated into the background image. This design is divided into three stages. Stage 1 handles the digitizing the signals and storing those signals to RAM as well as managing the access to the bus. Stage 2 handles the manipulation of the foreground image. Stage 3 handles the final output to the screen which involves combining the two images by position the foreground image onto the background image. The two video images are converted from analog to digital using a pair of AD775 A/D converter and GS4981 Sync Separator. Depending on user input, one signal is routed to foreground RAM and one signal is routed to the background RAM. Stage 2 has an intermediate RAM that is uses to calculate intermediate steps in it transformation and a ROM to provide the mathematical constants that it needs for it transform calculation. The output final output from Stage 2 is stored on fg_out RAM. Stage3 is constantly accessing background RAM and fg RAM to determine which pixel it should show. User input indicates it moving up and down. Access across to the databus needs to be managed across the three kits. Based on a 20 MHz clock, Stage 3 needs to output a pixel, every 2 clock cycles. Saving two pixels at time frees up a two clock cycle time period where the other two stages can access the bus. Stage 1 and 2 alternates between bus access times (See Figure 1).Figure 1: s3r is high during the times when Stage 3 has access to the data bus. s1w3 and s2w3 are low indicating when stage 2 and stage 3 have access to the data bus. Stage 1 Arbiter The purpose of the Arbiter is to manage access to the bus, upon which are control signals between the different stages and address and databits from the four RAMs and other control signal. The arbiter takes in three inputs and outputs four control signals. The threeinputs are bgrbar, which indicates that when Stage 3 cannot start reading from the bg RAM, midwbar which indicates when Stage 1 can write the foreground information to Stage 2, and fgwbar, which indicates when Stage 2 cannot start writing to the fg_out RAM. The outputs from the arbiter are timed to meet the timing convention already established for the bus. The four signals are s1w2 (Stage 1 can write to fg RAM), s1w3, (Stage 1 can write to bg RAM), s2w3 (Stage 2 can write fg out RAM) and s3r (Stage 3 can read from the RAMs). Because of limitations in the design, the inputs to the arbitrator are actually put on the bus. The arbiter only listens to those inputs when it is a valid time that Stage to have access to the bus. During the times when the stage doesn’t have access to the bus, the arbiter just holds the particular value from the last time that stage had access to the bus. Front The front module combines the digitizing and storing to RAM of the two video sources. It includes a storebg, storefg and sample clk module. There is a three sample clock cycle delay for the output from the AD 775 A/D converter so the appropriately delays the address from its internal so that they are sync correctly. In order to identify line and field information, a GS 4981 Sync separater was used to decode the sync signals. The front module processes these signals to identify odd and even fields. The front module also handles the user input that selects which video source should be used as background and which as foreground.storebg The storebg uses seven states to store video to the background RAM. It waits in the idle state until it gets a signal indicating the start of a field. It then goes to the wait-image state where it wait for the off the screen lines to pass. After the wait image state, it then moves to the wait sample state. In the wait sample state, storebg waits for a signal to tell it that the store a sample is available at which point it moves to the write sample state where it passes the data from the AD to one of two pixel registers. Storebg sends a signal indicating when it is writing on the first line of RAM to the Arbiter so it can in sure appropriate spacing between the addressing on the RAM. storefg The storefg also uses seven states to store video to the sample RAM. The process is very similar to the storebg FSM except that it stores an entire frame and then signals when it is done. Also before transitioning from the wait image state to the wait sample, it checks for write permission from the Arbiter which indicates Stage 2 is waiting data. If Stage 2 isn’t expecting data, it goes back to waiting for the beginning of the next frame. sampleclk The purpose of sample clock is to count the appropriate sample interval. It signals a sample interval every four clock cycles.Sync The purpose of this module is to sync the asynchronous inputs. The horizontal and vertical sync pulses as well as the reset button need to be sync to the clock. Stage 2 Overview of Image Processing Written by Vincent Wu The second stage of the Picture-in-Picture project involves performing scaling and rotation transforms on the foreground video feed. Conventional digital image processing involves multiplying an image represented as an array of points by a transformation matrix. However, for real-time digital video processing, this method of image processing is too costly for a real-time application. To transform our foreground video feed, the Stage 2 manipulates the stored image from Stage 1 with two passes, a vertical transform and a horizontal transform. The vertical transform performs operations on columns of the image, while the horizontal transform performs operations on rows of the image. The image can be processed as two one-dimensional transforms because of the separability of the transformation matrix referred earlier. This separability property applied to image processing was first proposed by Catmull and Smith 1. Stage 2 applies a similar two-pass approach proposed by Chen and Kaufman2.
View Full Document
Unlocking...