Statistical analysis and evaluation of spatiotemporal and compressed domains moving object detection Presented by Rajesh Radhakrishnan Instructor K R Rao Scope of the Project Three essential modules are required to obtain the comparative study of the moving object detection between two domains First is the manual annotation of hand locations using a GUI to get the co ordinate location of the object in every frame Second is to obtain time series of hand locations based on spatio temporal algorithm Third is to obtain time series of hand locations based on compressed domain algorithm Spatio temporal moving object detection Given a video test sequence we need to perform background subtraction to separate the foreground moving objects from the background model Fig 1 Block diagram of steps involved in spatio temporal object detection Background modeling is a process of obtaining static image regions from a sequence of video Frame differencing is one of the technique to perform background modeling Parametric and non parametric estimation model is a way to improve the candidate foreground object detection Parametric model Simple Gaussian model is an example of a parametric model Estimate parameters such as mean and standard deviation Consider a block of ground truth image and estimate the mean and standard deviation of the block Ground truth can be defined as the area under the actual moving object in each frame P color RGB P RGB color P color P RGB by Bayes rule Where P color RGB conditional probability and P x probability of x P RGB color is estimated from the training set which is the Gaussian probability of RGB color mean std where std standard deviation Assume P R P G P B are mutually independent then Gaussian probability of P RGB color is given by P RGB color P R color P G color P B color where P R color mean STD Gaussian probability R i j mean std where 1 i N 1 j M of an image of size NxM An example to find P RGB color Consider the sub block image shown in Fig 1 to estimate the mean and standard deviation of the green color object to be detected This is an sub block image of size 80x60 find the mean and standard deviation of each color band separately Fig 2 Sub block image to estimate mean and standard deviation How to estimate P color and P RGB P color can be of any value this determines the color adjustment factor Some of the prior probabilities are shown Prior Probabilities P x P x P x X X Fig 2 1 Prior probability distribution 4 Fig 2 2 Prior probability distribution 4 X Fig 2 3 Prior probability distribution 4 P RGB 1 as shown below P RGB P RGB color P color P RGB non color P non color Non parametric Estimation Non parametric model does not require any parameter estimates Histogram based distribution is one of the non parametric model For basic color object detection like red green and blue colors approximation based method can be done Suppose a green object has to be detected given a color image of dimension NxMx3 Subtract the green pixel region with the rest to get the probability of green distributions in the image Green dist i j 2 Image i j 2 Image i j 1 Image i j 3 Experimental Results Spatiotemporal object detection The spatio temporal moving object detection algorithm was tested on two video sequences one with a nearby object and another with a far end object Four set of output were generated with single and multiple detection boxes Object detection output of a frame with three detection boxes Fig 3 Close up video Frame 17 3 detection boxes Fig 4 Distant video Frame 19 3 detection boxes Object detection output of a frame with single detection box Fig 5 Close up video Frame 19 1 detection box Fig 6 Close up video Frame 17 1 detection box Compressed Domain Object detection Motion vector estimate is used to predict the moving object block Algorithm Rearrange the frames from bit stream order to display order Consider three pairs of arrays present past and future for storing the motion vectors The process of inputting the motion vectors into correct arrays and reordering frames were incorporated into the decoder Each video sequence is divided into one or more group of pictures GOPs the display order of the GOPs will be of the form given in fig 7 Here I B and P are intra coded bidirectional prediction and predicted frames Fig 7 MPEG group of pictures Display order 11 But the encoder output in bit stream order will be of the form I P B B P B B I B B P B B 11 Bit stream to display order conversion Converting from bit stream order to Display order Fig 8 Block diagram illustrating conversion from bit stream order to display order 11 If an P frame is encountered place it in a temporary storage called future P frame will be left in the future until another I or P frame comes in on arrival of a new I or P frame the already existing I or P frame is removed from the future and put in the display order All B frames are immediately put in display order Next step is to obtain the motion vectors from these frames Frame handling Program Operation Fig 9 Flow chart of the operational program 9 Each incoming frames are placed in a past present or future array locations based on their type i e either a P I or B frame The size of the array will be equal to the frame size in macro blocks i e the frame size used in this project is 240x320 for a motion vector of block size 8x8 array size would be 30x40 Once the motion vectors are stored the next step is to find the motion from frame to frame Finding motion from frame to frame The output of the present and past frame array motion vectors are used to find the motion from frame to frame Past Present I I P P B B or P I B or P I B or P Vector types that can be subtracted Forward only None Forward only None Forward and backward Table 1 Constraints to be taken into account For example consider a transition from B frame to a P or B frame it has both the forward and backward vector to be considered let a B frame macro block motion vector have values 4 6 for forward prediction and 6 1 for backward prediction Let a P frame macro block motion vector have values 9 7 for forward and 0 0 for backward as P frame doesn t have a backward prediction Total motion will be average of forward and backward prediction Forward 9 7 4 6 5 1 backward 0 0 6 1 6 1 The corresponding motion vector values are written into a file one for horizontal and another for vertical and its values were plotted using MATLAB The motion vector which gave a maximum direction was spotted …
View Full Document
Unlocking...