6.866 projectsDoes anyone mind…Today: Cameras looking at peopleYesterday’s tomorrowComputer vision still needs to become more robustBut we can fake it with clever system designResearch at MERL on fast, low-cost vision systemsComputer vision based interfaceExisting interfaces devices are fast & low-cost.Applications make the vision easier.There is a human in the loop.Computer vision algorithmsas ocean-going vesselsComputer vision algorithmsas ocean-going vessels1. Selected appliance: televisiontelevision marketSurveySurvey resultsControl of television setfrom a distanceDesign constraintsComplex commandsrequire complicated gestures?Living room scene is difficultOur solution: exploit the visual feedback from the televisionhand recognition method:template matchinghand recognition method:normalized correlationNormalized correlationBackground removalProcessing block diagramPrototype of television controlledby hand signals.TV screen overlayTV controlVideoPrototype limitationsProduct hardware requirements2. Simple gesture recognition methodReal-time hand gesture recognitionby orientation histogramsOrientation measurements (bottom) are more robust to lighting changes than are pixel intensities (top)Orientation measurements (bottom) are more robust to lighting changes than are pixel intensities (top)Images, orientation images, and orientation histograms for training setTest image, and distances from each of the training set orientation histograms (categorized correctly).Crane movements controlledby hand gesturesJanken gamevideoGames selected for vision interfaceImage moments give a very coarse image summary.Hand images and equivalent rectangles having the same image momentsArtificial Retina chip for detectionand low-level image processing.Artificial Retina chipArtificial Retina functionsFast image moment calculation with artificial retina chipGame: NightsMoment-based pointing controlMoment-based pointing controlGame: Magic CarpetMagic carpet game--figure analysis by hierarchical image momentsGame: DecathleteOptical-flow-based Decathlete figure motion analysisDecathlete 100m hurdlesDecathlete javelin throwDecathlete javelin throwvideoNintendo Game Boy CameravideoSummaryTo Trevor’s slides…oct29trevor.pdfPerceptive Context for Pervasive ComputingTrevor DarrellVision Interface GroupMIT AI LabPerceptually Aware DisplaysExample: A Face Responsive DisplayA Face Responsive DisplayTasks and Visual ModalitiesMode and Task MatrixFinding FeaturesFlesh color trackingColor ProcessingFlesh color trackingDetection with multiple visual modesCommon Detection Failure ModesRobust real-time performanceMode and Task MatrixA Key Technology: Video-Rate StereoVideo-rate stereoRGBZ inputRGBZ inputRGBZ inputVideo-Rate StereoRange feature for ID!Color feature for ID!Mode and Task MatrixRobust, Multi-modal AlgorithmSystem OverviewClassic Background Subtraction modelStatic Background Modeling ExamplesStatic Background Modeling ExamplesStatic Background Modeling ExamplesThe ALIVE SystemALIVEALIVE system, MITA Face Responsive DisplayVision-only Application:Interactive Video Effectsend6.866 projectsProposals to us by today. We will ok them by Oct. 31.3 possible project types:Original implementation of an existing algorithm.Rigorous evaluation of existing algorithm.Synthesis or comparison of several research papers.Proposals to us by today. We will ok them by Oct. 31.3 possible project types:Original implementation of an existing algorithm.Rigorous evaluation of existing algorithm.Synthesis or comparison of several research papers.Does anyone mind…If I use your photographed face for a simple face-detection demo program that we’ll run in class next time?If you do mind, please let me know (before Thursday).If I use your photographed face for a simple faceIf I use your photographed face for a simple face--detection demo program that we’ll run in class detection demo program that we’ll run in class next time?next time?If you do mind, please let me know (before If you do mind, please let me know (before Thursday).Thursday).Today: Cameras looking at peopleMIT 6.801/6.866Oct. 29, 2002MIT 6.801/6.866MIT 6.801/6.866Oct. 29, 2002Oct. 29, 2002A mini-application lecture: under controlled conditions (not general conditions), what human interaction applications can you build with the tools we’ve developed so far?To be compared with: more sophisticated detection, classification, and tracking tools that we’ll study over the rest of the course.Yesterday’s tomorrowNew York Worlds Fair, 1939New York Worlds Fair, 1939(Westinghouse Historical Collection)(Westinghouse Historical Collection)ElektroElektroSparkoSparkoComputer vision still needs to become more robustPavlovic, Rehg, Cham, and Murphy, Intl. Conf. Computer Vision, 1999But we can fake it with clever system designM. Krueger,“Artificial Reality”,Addison-Wesley, 1983.From MERL and Mitsubishi Electric:David Anderson, Paul Beardsley, Chris Dodge, William Freeman, Hiroshi Kage, Kazuo Kyuma, Darren Leigh, Neal McKenzie, Yasunari Miyake, Michal Roth, Ken-ichi Tanaka, Craig Weissman, William YerazunisFrom MERL and Mitsubishi Electric:From MERL and Mitsubishi Electric:David Anderson, PaulDavid Anderson, PaulBeardsleyBeardsley, , Chris Dodge, William Freeman, Hiroshi Chris Dodge, William Freeman, Hiroshi KageKage, Kazuo , Kazuo KyumaKyuma, Darren Leigh, Neal , Darren Leigh, Neal McKenzie,McKenzie,YasunariYasunariMiyake, Miyake, MichalMichalRoth, Roth, KenKen--ichiichiTanaka, Craig Tanaka, Craig WeissmanWeissman, , William William YerazunisYerazunisResearch at MERL on fast, low-cost vision systemsComputer vision based interfaceThe hope: video input will give a more The hope: video input will give a more expressive, natural or engaging interface.expressive, natural or engaging interface.Existing interfaces devices are fast & low-cost.Applications make the vision easier.Constraints simplify recognition--if you know where the tracks are, it’s easy to guess where the train is.There is a human in the loop.z Rich, immediate visual, audio feedback.z The player can correct for algorithm imperfections.Computer vision algorithmsas ocean-going vesselsComputer vision algorithmsas ocean-going vesselsthiswork1. Selected appliance: televisiontelevision market ~1 billion television sets~1 billion television setsSurvey“What high technology gadget has improved the “What high technology gadget has improved the quality of your life the most?”quality of your life the