Digital Video Processing (EE392J)Department of Electrical EngineeringStanford UniversityProblem Set No. 4Issued: Monday, February 12, 2007Due: Wednesday, February 21, 2007Reading: Chapter 4Office hours: Mon (2/12) and Wed (2/14) after class, Fri (2/16) 2:00-3:00 (2nd floor lounge in Packard Building).Please feel free to stop by office hours to talk about possible projects.Problem 1. Problem 4.2. If you prefer, you can hand-sketch the magnitude of the frequency response instead of usingMatlab.Problem 2. An important area of video processing today is frame rate conversion. An example of this can be seen inour current television system. Motion pictures are typically shot at 24 frames/sec while our television system is basedon the transmission of 60 fields/sec. An “upsampling” process is required to couple the 24 frames/sec film to the 60fields/sec transmission format. To simplify matters, let us assume that it would be sufficient if the upconverted signalwas at 60 frames/sec.Given the original continuous-time video signal, this could be accomplished by sampling at a rate 2.5 times ashigh as before. However this is not a practical approach. Instead a technique called 3:2 pulldown is typically used(Figure 1). In 3:2 pulldown the first frame is repeated 3 times, then the second frame is repeated 2 times, the thirdframe is repeated 3 times, etc. Each odd numbered frameis repeated 3 times and each even numbered frame is repeatedtwice. 1 2 3 4 5Original signal(24 frames/sec)Frame number 1 1 2 1 2 3 3 3 4 4 5 5 5Upconvertedusing 3:2 pulldown(60 frames/sec)Frame numberFigure 1: 3:2 pulldownA. How can we model this 3:2 pulldown process from a DSP point of view. That is, how can we represent thisprocess using upsamplers, downsamplers, and linear filters. Draw an appropriate block diagram, specifying theimportant elements. Make sure to specify the intermediate lattice, and the equivalent linear filter applied on thatlattice.B. What are the artifacts typically seen in 3:2 pulldown? What produces these artifacts? (Be brief.)(see back)Improved frame rate conversion can be achieved by carefully examining the spatio-temporal characteristics of avideo signal. For example, if we have a continuous-time video signal s(x, y, t) that is stationary (i.e., no motion) thenwhen we examine its 3-D spatio-temporal Fourier transform S(Fx,Fy,Ft) all of the energy is concentrated in theS(Fx,Fy, 0) plane, i.e., in the plane formed by Ft=0. If the motion is in the form of a constant global shift fromone frame to the next (global motion with constant velocity for the continuous video) the plane containing the energybecomes tilted in the 3-D Fourier domain. If the motion is not exactly in the form of a global shift, the energy spreadsout around this tilted plane. Note that this can be an appropriate model for local regions of the video signal.Continuous-Time Continuous-Space SignalF-24 Hz 24 Hz 48 Hz-48 Hz -24 Hz 24 Hz 48 Hz tFxF-48 HztxFx1Spatial-Temporal Fourier Transform of theDiscrete-Time Discrete-Space SignalSpatio-Temporal Fourier Transform of theFigure 2: Spatio-temporal Fourier transform of sampled signal.Assume that there is no motion along the y-direction, this allows us to restrict our analysis to the (Fx,Ft) plane.If we have a motion picture moving along the x-direction with global motion with constant velocity, and sampled at24 frames/sec, the support of its spatio-temporal Fourier transform is as shown in Figure 2. Note that the replicationsof the motion picture’s baseband appears at multiples of 24 Hz.C. Can the original continuous video signal be reconstructed by applying a separable continuous filter? That is, byapplying a filter with impulse response h(x, y, t)=hx(x)hy(y)ht(t)? If yes, explain. If not, why not, and whattype of filter may be used? In either case please be explicit describing the filter’s passband and stopband, andhow this filter support may be achieved.D. Consider an arbitrary television video signal (assume progressive scanning at 60 Hz), that is subsequently sam-pled at 24 Hz (frames/sec). Can we prefectly reconstruct the 60 Hz video signal such that it is identical to if wehad sampled the video at 60 Hz? If yes, briefly explain. If no, give counterexample.Problem 3. Project ProposalSubmit a one or two page proposal describing the final project that you will like to do for this course. Examplefinal projects are described on the class web page. Additional projects involving video processing or the processingof multiple images may also be possible – feel free to propose relevant alternate topics/problems that interest you. Aproject may be performed by a single student or a group of two or three students; see the class web page for furtherdetails. We will provide a number of video test sequences which may be useful for different projects.The project proposal should include the following:1. Name and description of project2. Individual project or group project?3. Why is this project interesting?4. What problem(s) must be overcome in this project?5. What methods are you considering to solve these problem(s)?6. How familiar are you with these problems?7. What useful references have you found in regard to these problems? Do you need additional references?8. Is additional equipment necessary (besides a computer)? If so, do you have access to this equipment or do youneed help getting access to this equipment?The ideal goal is to find a project that is both highly educational and fun. We’re available to meet and discuss anypossible projects, as well as to help you identify potential problems within the project and approaches to overcomethese problems.Office hours: Office hours are listed on the first page. Please feel free to stop by office hours to talk about