DOC PREVIEW
Berkeley COMPSCI 294 - Local Affine Feature Tracking in Films/Sitcoms

This preview shows page 1-2-3-4-5-6 out of 19 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 19 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Local Affine Feature Tracking in Films/SitcomsObjectiveOutlinePreprocessingTracking AlgorithmSlide 6Slide 7Slide 8Slide 9Slide 10Shot grouping/Scene RetrievalInter-Shot Matching“Confusion Table”ROCWhen Does Tracking Fail?Computation ComplexityConclusionFuture WorkAcknowledgementLocal Affine Feature Tracking in Films/SitcomsChunhui GuCS 294-6Final PresentationDec. 13, 2006Objective•Automatically detect and track local affine features in film/sitcom frame sequences.–Current Dataset: Sex and the City–Why sitcom?•Simple daily environment•Few or no special effects•Repeated scenesOutline•Preprocessing•Tracking Algorithm–Pairwise local matching–Robust features•Feature Matching across Shots•Results–Feature matching vs baseline color histogram–Time complexity–When does tracking failPreprocessingFrameExtraction(i-1)’th shot i’th shotShotDetectionMSER Interest PointDetectionSIFT FeatureExtractionTracking Algorithm•Basic: Pairwise MatchingFrame i Frame j=i+1imf( ),i im mx yjnfTracking Algorithm•Basic: Pairwise MatchingFrame i Frame j=i+1imf( ),i im mx yjnfTracking Algorithm•Basic: Pairwise MatchingFrame i Frame j=i+1imf( ),i im mx yjnf{ }minfdThresholding on both minimum distance and ratioTracking Algorithm•Basic: Pairwise MatchingFrame i Frame j=i+1imf( ),i im mx yjnfTracking Algorithm•Basic: Pairwise MatchingFrame i Frame j=i+1imf( ),i im mx yjnfTracking Algorithm•Problem of Pairwise Matching–Sensitive to occlusion and feature misdetection•Solutions:–Use multiple overlapping windows–Backward Matching •Match features in current frame to features in all previous frames within the shot•Pruning process (reduce computation time)•Select a proportion of features that have longer tracking length as robust featuresShot grouping/Scene Retrieval( )60601 2, ,...rf mf x x x( )56561 2, ,...rf mf x x x10746 10747 10772Shot 4910933 10934 10968Shot 5311393 11394 11435Shot 56Shot 6011533 11534 11560Scene 5( )49491 2, ,...rf mf x x x( )53531 2, ,...rf mf x x xInter-Shot MatchingShot IShot J( )1 11 2, ,...I mf x x x( )2 21 2, ,...I mf x x x( )1 11 2, ,...J nf x x x( )2 21 2, ,...J nf x x x( )1 2, ,...q qJ nf x x x( )1 2, ,...p pI mf x x xD“Confusion Table”Ground Truth50 55 60 65 70 75505560657075Color Histograms50 55 60 65 70 75505560657075Feature Matching50 55 60 65 70 75505560657075ROC0 0.2 0.4 0.6 0.8 100.10.20.30.40.50.60.70.80.91False AlarmTrue DetectionROC curve of Feature MatchingWhen Does Tracking Fail?•Tracking feature outside local window–Rare when continuous tracking–Happens when occlusion occurs•Same feature splitting to two or more groups–Long occlusion–Multiple matching in a single frameFrame i Frame j=i+1imf( ),i im mx yjnfComputation Complexity•Everything except for MSER and SIFT algorithms are implemented in Matlab (slow…)Complexity TimeFrame Extraction O(N) ~0.3s/frameShot Detection O(N*f(B)) ~0.07s/frame (B=16)MSER Detection O(N) ~0.3s/frameSIFT Detection O(N) ~0.9s/frameFeature Tracking O(N*F*W*L) ~0.5s/frameMatching across shotsO(S2*T2) ~1s/shot pairN: # of frames; (30,000) B: # of bins for color hist (16) F: ave. # of features per frame; (400) W: Local window size; (15)L: tracking length; (20) T: ave. # of robust trackers per shot; (300)S: # of shots; (35)Conclusion•We successfully implemented local affine feature tracking in sitcom “sex and the city”. The tracking method is robust to occlusion and feature misdetection.•Although no quantitative precision/recall curve (hard to find ground truth), the demonstration shows that precision is almost perfect with good recall performance.•We show one successful application of using robust features to associate similar shots together for scene retrieval.Future Work•Implement algorithm in real-time (C/C++)•Search unique shots in films/sitcoms•Separate indoor scenes from outdoor scenes•Determine context of the


View Full Document

Berkeley COMPSCI 294 - Local Affine Feature Tracking in Films/Sitcoms

Documents in this Course
"Woo" MAC

"Woo" MAC

11 pages

Pangaea

Pangaea

14 pages

Load more
Download Local Affine Feature Tracking in Films/Sitcoms
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Local Affine Feature Tracking in Films/Sitcoms and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Local Affine Feature Tracking in Films/Sitcoms 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?