Group Report Real Time Color Replacement ECE 532 design project 2009 Chang Liu (995416250) Andrei Bajenov (995478763) Jeffery Sham (995384310) April 9, 20092 | P a g e Table of Contents PAGE 1. Overview .............................................................................................................. 3 1.1 Goals ............................................................................................ 3 1.2 Project Background ..................................................................... 3 1.3 Invisibility Effect ......................................................................... 3 1.4 System Block Diagram ................................................................ 5 1.5 Brief Description of IP ................................................................ 5 1.6 Clock Domain ............................................................................ 7 2. Project Outcome ................................................................................................. 7 2.1 Review of Original Plan ............................................................... 7 2.2 Our Final Product ........................................................................ 8 2.3 Suggestions For the Future ......................................................... 8 3. Description of Blocks ....................................................................................... 9 3.1 MicroBlaze .................................................................................. 9 3.2 PLB Bus ..................................................................................... 10 3.3 IIC Interface ............................................................................. 11 3.4 TFT Controller .......................................................................... 11 3.5 MPMC Controller ..................................................................... 12 3.6 Decoder Logic .......................................................................... 12 3.7.1 IP_to_Mem Custom Logic ........................................................ 12 3.7.2 Color Detection Algorithm ....................................................... 15 3.8 Mem_to_IP Custom Logic ........................................................ 16 3.9 Connection between IP_to_Mem and Mem_to_IP .................... 18 4. Description of Our Design Tree ....................................................................... 19 5. Reference ......................................................................................................... 20 6. Appendix .......................................................................................................... 21 6.1 PLB Burst Write Timing Diagram ............................................ 21 6.2 PLB Burst Read Timing Diagram ............................................. 22 6.3 PLB Single Beat Read Timing Diagram ................................... 233 | P a g e OVERVIEW 1.1 Goals The goal of our project was to develop a system which performs real-time color replacement of a video stream. In particular, our system will detect a certain colored pixel in the video frames and alter it with a pixel from a different image in the video output. Using this concept, we are not only able to implement a simple commercial Chroma Key using only hardware, but to also develop a device that can make you completely invisible on camera! 1.2 Project Background Chroma keying, more commonly known as “green screening”, is a technique used in movie production to film scenes that are too difficult to achieve with props, set pieces, and other special effects. In Chroma keying, the actor performs a scene in front of a coloured screen and later, the specified colour is filtered out and replaced with the desired background. There are a number of subtleties involved in perfecting Chroma keying that makes it a tedious process. This includes correctly defining the colour to key out in order to avoid “colour bleeding” and applying even lighting on the backdrop and subject to achieve the best contrast. However, for the purposes of our application, we are more interested in demonstrating that the process can be done in real-time, and will accept the slightly lower quality result given the limitations of our lighting and backdrop equipment. Thus, our goal is to be able to take in video frames from the camera, detect the pixels with a backdrop colour, and replace them with an image that is saved in memory. In order to satisfy the real-time criteria, the processing should be fast enough that when the foreground (subject) changes, there is no detectable glitch in the video output. 1.3 Invisibility Effect The trick behind the invisibility effect is taking a still shot of the background before the subject enters the set and replace the desired colour with the saved image to create the illusion that the subject has disappeared. This requires that the camera does not move and that the background is not changed. Failure to do so will cause discrepancies between the saved background and actual background, thus ruining the effect. In terms of the technical aspect, we first need to be able to save the background as a static frame somewhere in our memory. This will allow to know exactly what the background is at all times during the live stream even when something or someone is blocking it. There are key similarities in the implementation of Chroma key and the4 | P a g e invisibility effect that allowed us to achieve both with very little addition. In the original Chroma keying concept, the static image comes from non-volatile memory such as compact flash. In the invisibility effect, the static frame comes from the initial frame input from the camera. In both cases, we would need to use memory for the frame and we must create our hardware so that it will read in lines of data for the pixel replacement. The slight extension in implementing the invisibility effect is the extra logic required in our hardware to save the initial frame into DDR memory. However, it should be noted that if our goal was only to replace one colour with another colour, we can simply modify the video stream as it streams to VGA and bypass one of our