6.111 Final Project Proposal Wolfe Styke Gesture Recognition Remote Control Overview The Gesture Recognition Remote Control system aims to provide a new method for controlling a television. Today, remotes control televisions entirely through the pressing of buttons. At the beginning of the development of video game consoles, video games were controlled entirely through button presses, very much like televisions are today. This is no longer the case for game consoles; consider the Wii Remote, which uses the position of the controller in addition to button presses to control a video game. This project aims to bring this gesture capability to the effective control of a television. Project Description The system will consist of two main parts. The remote device, hereafter called a wand, which the user interacts with directly, and the control box consisting of the FPGA device connected to input sensors and output transmitters which the user interacts with through the remote device. The Remote Device The wand will be designed to be as easy to use as possible. On the end of the wand will be several LEDs of one color. The user turns these LEDs on when making a gesture, by pressing and holding a button on the wand. The user will have two wands, with different colored LEDs at the end, to increase the number of gestures he or she may use to control the television. The Control Box The control box will be responsible for turning the movements of the wands into the appropriate infrared signals that a television can recognize as specific commands (such as volume up). There will be two main inputs to the system, and one main output. Inputs/Outputs One of the inputs, the video camera, will be used by the control box to detect the presence of the wands when their LEDs are turned on. The other input, an infrared receiver that filters for frequencies used by conventional remote controls, will be used by the control box to gather the signal information of specific commands output by a particular remote control. Time permitting; this input could be used by the system to gather information about the signals needed to control a different television than the one which the control box was originally programmed to control. As an output, an infrared transmitter similar to those used by standard remote controls would be used by the control box to send signals to the television similarly to how standard remote controls send signals.The Internals The control box internals will need to turn video input to IR signal output for the primary functionality of the device. It will also need to store IR signal input in memory to be called upon later by the primary functionally. Analyzing the Video Input When the wand is detected, the information of the wand’s location will be passed through a gesture recognition system, which will determine which gesture the user is making with the wands. The video signal coming in to the control box will first be stored in ZBT memory. Then the image will be analyzed from the memory to find the location of the pixels showing the two wands. These pixels for both wands will then be sent through their own center of mass algorithm to find the coordinates of both wands. When the wand is detected, the information of the wand’s location will be passed through a gesture recognition system, which will determine which gesture the user is making with the wands. By sampling the center of mass coordinates of the two wands several times a second, until the LEDs on the wands are turned off, a gesture recognition module will be able to determine which gesture the user is making. This gesture will then be passed to a command module which will reference command signals stored in memory, which will be sent to the infrared transmitter to control the television. Acquiring New Signals for Commands This functionality would allow for flexibility of use of the overall system, removing the restriction that the system be used only with the television system for which it was initially programmed. A button on the control box could be pressed to initiate a “programming session” during which the user would press buttons, in a particular sequence, on the remote control he or she would like to emulate with the control box. The signals from the remote would be picked up by an infrared receiver, which would update the memory storing the old signal code associated with each particular command with the new signal code, such that later when the user makes gestures, the new signal code is referenced and sent to the infrared transmitters. Testing The modules for each part of the conversion of video input to infrared signal output will be connected to switches for diagnostic testing. Some switches will determine which module is being diagnosed, while another set of switches will provide commands to that module. The outputs of each module will also be redirected to either the 16-digit hex display or an external monitor for verification of correct behavior. The coordinate extraction modules can be tested by one of two methods. The pixels of the camera detecting a wand can be passed out to an external monitor similar as in lab 5, showing the spheres of color at the end of each wand. Another method to test these modules would be to show the coordinates of the two wands’ center of mass on the 16-digit hex display. Once the coordinate extraction modules work, the gesture recognition modules can be tested by displaying the gesture recognized on the 16-digit hex display. To test the modules responsible for using the gesture todecide which signal to send to the infrared transmitter, the four arrow keys on the lab kit can be utilized to specify a direction while the code that the modules look up based on that direction can be displayed on the 16-digit hex display. This mode of testing can also be used to verify that the system properly received and recorded in to memory a signal from a new remote control. Figure 1. This block diagram shows the major modules processing the v ideo and infrared signal inputs into infrared signal outputs and updated memory information respectively.Figure 21 above. This depicts the signal of an rc-5 infrared signal that is used currently by some remotes to control televisions. The address bits select the device, television one and two are assigned addresses of zero and one respectively. The most significant bits of the address and command are
View Full Document
Unlocking...