Slide 1Problems AddressedExampleThe Semantic GapOur ApproachEvent RepresentationEvent ComparisonEvent DetectionLabeled Video EventsSlide 10Experiment #1Slide 12Slide 13Slide 14Slide 15Classifying Disguised EventsSlide 17Slide 18Slide 19Slide 20XML Video AnnotationVideo Analysis ToolSummary and DirectionsAccess Control and BiometricsPrivacy Preserving Surveillance - IntroductionSystem UseSystem ArchitectureAcknowledgements19-1Data Mining for Surveillance ApplicationsSuspicious Event DetectionDr. Bhavani ThuraisinghamNovember 200719-2Problems AddressedProblems AddressedHuge amounts of video data Huge amounts of video data available in the security domainavailable in the security domainAnalysis is being done off-line Analysis is being done off-line usually using “Human Eyes”usually using “Human Eyes”Need for tools to aid human Need for tools to aid human analyst ( pointing out areas in analyst ( pointing out areas in video where unusual activity video where unusual activity occurs)occurs)Consider corporate security for a Consider corporate security for a fenced section of sensitive fenced section of sensitive propertypropertyThe guard suspects there may The guard suspects there may have been a breach of the have been a breach of the perimeter fence at some point perimeter fence at some point during the last 48 hoursduring the last 48 hoursThey must:They must:Manually review 48 hours of tapeManually review 48 hours of tapeConsider multiple cameras and Consider multiple cameras and camera anglescamera anglesDistinguish between normal Distinguish between normal personnel and intruderspersonnel and intruders19-3ExampleExampleUsing our proposed system:Using our proposed system:Greatly Increase video analysis Greatly Increase video analysis efficiencyefficiencyUser DefinedEvent of interestVideo DataAnnotated Video w/ events of interest highlighted19-4The Semantic GapThe Semantic GapThe disconnect between the low-level The disconnect between the low-level features a machine sees when a video is features a machine sees when a video is input into it and the high-level semantic input into it and the high-level semantic concepts (or events) a human being sees concepts (or events) a human being sees when looking at a video clip when looking at a video clip Low-Level featuresLow-Level features: : color, texture, shapecolor, texture, shape High-level semantic conceptsHigh-level semantic concepts: : presentation, newscast, boxing matchpresentation, newscast, boxing match19-5Our ApproachOur ApproachEvent Representation Event Representation Estimate distribution of pixel intensity Estimate distribution of pixel intensity change change Event ComparisonEvent ComparisonContrast the event representation of Contrast the event representation of different video sequences to determine if different video sequences to determine if they contain similar semantic event content.they contain similar semantic event content.Event DetectionEvent DetectionUsing manually labeled training video Using manually labeled training video sequences to classify unlabeled video sequences to classify unlabeled video sequences sequences19-6Event RepresentationEvent RepresentationMeasures the quantity and type of changes occurring Measures the quantity and type of changes occurring within a scene within a scene A video event is represented as a set of x, y and t intensity A video event is represented as a set of x, y and t intensity gradient histograms over several temporal scales.gradient histograms over several temporal scales.Histograms are normalized and smoothedHistograms are normalized and smoothed19-7Event ComparisonEvent ComparisonDetermine if the two video sequences Determine if the two video sequences contain similar high-level semantic concepts contain similar high-level semantic concepts (events). (events). Produces a number that indicates how close Produces a number that indicates how close the two compared events are to one another. the two compared events are to one another. The lower this number is the closer the two The lower this number is the closer the two events are. events are. 221 2, ,1 2[ ( ) ( )]13 ( ) ( )l lk kl lk l ik kh i h iDL h i h i-=+�19-8Event DetectionEvent DetectionA robust event detection system A robust event detection system should be able toshould be able toRecognize an event with reduced Recognize an event with reduced sensitivity to actor (e.g. clothing or skin sensitivity to actor (e.g. clothing or skin tone) or background lighting variation.tone) or background lighting variation.Segment an unlabeled video containing Segment an unlabeled video containing multiple events into event specific multiple events into event specific segmentssegments19-9Labeled Video EventsLabeled Video EventsThese events are manually labeled These events are manually labeled and used to classify unknown eventsand used to classify unknown eventsWalking1 Walking1 Running1Running1Waving2Waving219-10Labeled Video EventsLabeled Video Events##walkinwalking1g1walkinwalking2g2walkinwalking3g3runninrunning1g1runninrunning2g2runninrunning3g3runninrunning4g4waving waving 22walkinwalking1g1000.276250.276250.245080.245081.22621.22621.3831.3830.974720.974721.37911.379110.96110.961walkinwalking2g20.276250.27625000.178880.178881.47571.47571.50031.50031.29081.29081.5411.54110.58110.581walkinwalking3g30.245080.245080.178880.17888001.12981.12981.09331.09330.886040.886041.12211.122110.23110.231runninrunning1g11.22621.22621.47571.47571.12981.1298000.438290.438290.304510.304510.398230.3982314.46914.469runninrunning2g21.3831.3831.50031.50031.09331.09330.438290.43829000.238040.238040.107610.1076115.0515.05runninrunning3g30.974720.974721.29081.29080.886040.886040.304510.304510.238040.23804000.204890.2048914.214.2runninrunning4g41.37911.37911.5411.5411.12211.12210.398230.398230.107610.107610.204890.204890015.60715.607wavingwaving2210.96110.96110.58110.58110.23110.23114.46914.46915.0515.0514.214.215.60715.6070019-11Experiment #1Experiment #1Problem: Recognize and classify events Problem: Recognize and classify events irrespective of direction (right-to-left, left-irrespective of direction (right-to-left, left-to-right) and with reduced sensitivity to to-right) and with reduced sensitivity to spatial variations (Clothing)spatial variations (Clothing)““Disguised Events”- Events similar to Disguised Events”- Events similar to
View Full Document
Unlocking...