ESC532 Final Design Report Real Time Note Digitizer Konstantine N J Tsotsos Sanae L M Rosen Sean C Bell Team A 350 3 4 April 2011 1 Contents 1 Overview 4 1 1 Goals 4 1 2 Block Diagram 5 1 3 Brief Description of IP 6 1 3 1 Reference Design 6 1 3 2 Processor Code 7 1 3 3 Video to ram 7 1 3 4 Other IP 8 2 Outcome 2 1 2 2 8 Results Possible Further Improvements 3 Description of the System Components 8 10 10 3 1 video to ram 10 3 1 1 Decision Tree Classifier 11 3 2 Processor 12 3 2 1 Memory map 12 3 2 2 VGA h 12 3 2 3 History 13 3 2 4 Menu System 14 3 2 5 Replay 15 Internal Communication and External Ports 15 3 3 1 Video input 15 3 3 2 Internal Communication 15 3 3 3 Video output 15 3 3 4 Description of Design Tree 16 Appendices 17 A Experimentation with Classification Schemes 17 B Previous design all processing in hardware 19 2 List of Figures 1 Projector1 2 Block diagram of system showing internal and external connections 5 3 Overview of IP 6 4 Final decision tree for system 11 5 Memory map for SDRAM accessed by both the processor and the video to ram module 13 6 Design of system for processing entirely in hardware 19 4 List of Tables 1 Status of initially proposed features 9 2 Status of initially proposed functional requirements 9 3 Status of initially proposed acceptance criteria 9 4 Split on Cb 17 5 Split on Cr 17 6 Split on Y 18 3 1 Overview 1 1 Goals The goal of this project is to create a tool to aid in digitizing handwritten notes The project was motivated by instructors who display notes using overhead projectors While students can see the final process a compact digital representation for playback is generally not available Merely recording a video of the notes is not satisfactory as this includes unncessary visual components such as the instructor s hand Our system provides a compact representation of notes being written that removes clutter and reduces file size The system records only the notes written on a piece of paper removing the user s hand and other extraneous data and displays them via a VGA port Figure 1 Projector1 The system is robust to movement of the paper and allows the user to write in multiple colours To implement this each pixel is classified in hardware as ink or not with a decision tree classifier The classifier is designed to further exclude pixels having a skin colour The output of the classifier is written to RAM to be later read by the processor The processor computes the time that each pixel has been classified as ink and uses this information to generate video output The entire process is recorded for later playback 1 Image adapted from http www instructables com id How to Make XXL Street Stencils 26 Get AwayWIth It step5 Scale up your letters 4 1 2 Block Diagram A block diagram of the system is shown in Figure 2 The functionality of each block is described in later sections 9 3 8 6 6 01 7 3 012 3 012 3 4 56 4 7 5 73 3 1 7 E C0 4ABA D I3 5 3 J6 5 K17L17 3 5 FG 5 E FG7H 37 0 D 3 E 7 3M 6 7 Figure 2 Block diagram of system showing internal and external connections 5 1 3 Brief Description of IP Figure 3 Overview of IP 1 3 1 Reference Design As a basis for the video processing portion of our design we used a reference design by Jeffrey Goeders posted on the 2011 discussion board on Blackboard for ECE532 1 In the reference design the video to ram pcore reads in data from a video camera using the video decoder It converts from YCbCr format to RGB format and sends pixels to RAM via the PLB bus All control signals for timing is computed in video to ram There is a second pcore video out which reads pixels from RAM and generates signals necessary to drive the monitor via VGA The video decoder is 1 https portal utoronto ca E163B5E40E942455D203314EA00BDC27 courses 1 Winter 2011 ECE532H1 S LEC0101 db 2337128 1 Video To RAM System rar 6 configured using the processor via IIC The code for configuring the video decoder is contained in video setup c We adapted the design for our project and modified modules contained in the video to ram pcore as well as the code in video setup c 1 3 2 Processor Code The primary source file is video setup c Originally this code was borrowed from the Video to RAM project by Jeffrey Goeders and only configured the video decoder We added in additional functionality as follows It receives data from the video to ram module and based on the values given either 1 or 0 for black or white stores these in a buffer When it determines that a pixel has been black for a period of time it decides it corresponds to ink and not to a transient shadow and sets the pixel value accordingly When it determines that a pixel has been white for a longer period of time it decides that the ink has disappeared and is not occluded by a hand and removes it It stores frames in a circular buffer so that it is possible to replay a video of the notes being written A menu allows the user to pause recording replay the buffer or change the colour of the ink 1 3 3 Video to ram This hardware module does the following things It reads in data from the video camera This functionality is based on the pcore Video To RAM by Jeffrey Goeders This data is classified using the YCbCr colour data for each pixel The hand trained decision tree classifier first uses the Cr value to decide if a pixel belongs or does not belong to a hand For the non hand pixels it then classifies them as ink or paper based on the Y value It converts the decision made for each pixel to a RGB value either all ones or all zeros i e either black or white and sends one bit per value per pixel to the processor 7 1 3 4 Other IP dlmb ilmb dlmb cntlr ilmb cntlr Used for communicating between the processor and the BRAM lmb bram The block RAM plb uB The master on this bus is the processor which uses it to communicate with the video to ram module the UART module and the SDRAM primarily The iic and debug modules are also slaves on this bus plb video out The TFT module which communicates with the VGA is the master on this bus which uses it to communicate with the SDRAM This is used to communicate between the processor and the tft module that communicates with the VGA plb video to ram This is used …
View Full Document