Panoramic Video from Single Video Clip Michael Chuang Carnegie Mellon University ABSTRACT – This paper presents a method to use a single video clip and show the foreground object’s movement in a timeline fashion. The end result implementation will give the viewers a perspective of the foreground object’s motion over the entire background image. INTRODUCTION It is very common in entertainment and sporting events to see the event through one camera angle like skating. Through the narrow camera view, sometimes it is hard to see an athlete’s movement over the entire background. It would be really nice to overlay a player’s movement over a certain timeline to see how an athlete performed or just to get a bigger perspective of the event. In this paper, we implement an approach to solving this problem and show our preliminary results. RELATED WORKS Several related works have been implemented for the entertainment and sports industry. Since the implementation of our approach used a miniature football field, we highlight a few of the more popular products used by the sports industry. A European company named Dartfish has created a software program that we have described for the Olympics. Their software tool overlays a skater’s movement over the background, which we show in figure 1. Another sports-based company called SportVision has done something similar with soccer. They overlay a shot’s trajectory when a player scores a goal show in figure 2. They also are able to overlay a quarterback’s pass after he throws it for football. Figure 2. StroMotion (SportVision) . Figure 1. Ice Skater (Dartfish)APPROACH The Process: 1. Get input video clip 2. Separate into individual frames 3. Stitch images together to form background 4. Separate foreground from global background for each frame by subtracting images 5. Filter out points with threshold 6. Further refine points with 4-connector filter 7. Improve foreground output by dilating remaining pixels 8. Overlay each foreground image onto global background to create each movie clip frame 9. Combine frames into video clip. RESULTS In our results section, we walk the readers through our approach and implementation output. Figure 3. Sample Video Clip Frame After separating the video clip to frames (we show a sample frame in figure 3), we stitch the images to take a global background which we show in figure 4. We then take the difference between the current frame and the global background which is shown in figure 5. However, if you notice there are many single pixels that show up because of various minor reasons such as lightening differences. We try to solve the resulting problem by using a 4-connector filter in MATLAB called BWareaopen. As you can see from the results in figure 6, it removes a lot of the points we don’t need. Figure 4. Background Figure 5. Difference Between Figure 3 and Background Figure 6. Refine Points with 4-connector filterFrom figure 6, if we try to use it as a mask and copy the foreground object’s pixels over the global background, a ghosting-type situation occurs because there were holes in the mask (shown in figure 7). Figure 7. Ghosting Figure 8. Dilating Remaining Pixels We solve the ghosting problem by dilating the remaining pixels in MATLAB through a function called imdilate. This gives us nice looking blobs. As you can see from figure 9, the players look a lot better now. It isn’t perfect for some frames as you can still see some of the helmets are missing. We tried playing around with the thresholds and these were one of the better results. - Difference Threshold = 10 - At least 4-Connector for Bwareaopen - Dilate = 5 pixels Figure 9. Resulting Overlay Timeline CONCLUSION Through our preliminary results, we have shown a method of creating a timeline of an object’s movement over a background. We are interested in improving this implementation setup to be able to handle moving cameras. REFERENCES [1] H.Y. Shum, R. Szeliski. Panoramic Image Mosaics. Microsoft Research Tech Report 1997 [2] www.dartfish.com [3]
View Full Document
Unlocking...