CS6670: Computer VisionInstructorOther detailsTodaySlide 5AnnouncementEvery image tells a storyThe goal of computer visionCan the computer match human perception?Human perception has its shortcomingsBut humans can tell a lot about a scene from a little information…Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Why study computer vision?Optical character recognition (OCR)Face detectionSmile detection?Object recognition (in supermarkets)Face recognitionVision-based biometricsLogin without a password…Object recognition (in mobile phones)iPhone Apps: (www.kooaba.com)Special effects: shape captureSpecial effects: motion captureSpecial effects: camera trackingSportsSmart carsVision-based interaction (and games)Vision in spaceRoboticsMedical imagingMy own workPhotosynthCity-scale reconstructionCurrent state of the artWhy is computer vision difficult?Slide 48Challenges: local ambiguityBut there are lots of cues we can exploit…Bottom lineCourse overview (tentative)1. Low-level visionProject 1: Feature detection and matching2. GeometryProject 2: Creating panoramas3. RecognitionProject 3: Recognition challenge (TBA)4. Light, color, and reflectance5. Advanced topics: Internet VisionFinal projectCourse requirementsGrading3-minute breakCS6670: Computer VisionNoah SnavelyInstructor•Noah Snavely ([email protected])•Office hours: Wednesdays 10:30 – noon, or by appointment(plus this Friday at 3-4pm)•Research interests:–Computer vision and graphics–3D reconstruction and visualization of Internet photo collectionsOther details•Textbook:R. Szeliski, Computer Vision: Algorithms and Applicationsonline at:http://research.microsoft.com/en-us/um/people/szeliski/Book/(please check Web site periodically for updated drafts)•Course webpage: http://www.cs.cornell.edu/courses/cs6670/2009fa/•Mailing list: [email protected]. Introduction to computer vision2. Course overview3. Basic image processing•Note: Class will not meet next week–Next meeting: Tuesday, September 8Today•Readings–Szeliski, CV: A&A, Ch 1.0 (Introduction)•Handouts–signup sheet–intro slides–image filtering slidesAnnouncement•Today: Conway-Walker Distinguished LectureAndy Wilson, Microsoft Research on Surface ComputingToday, 4:15pm (right after class)B17 Upson HallEvery image tells a story•Goal of computer vision: perceive the story behind the picture•Compute properties of the world–3D shape– Names of people or objects–What happened?The goal of computer visionCan the computer match human perception?•Yes and no (but mainly no, so far)–computers can be better at “easy” things–humans are much better at “hard” thingsHuman perception has its shortcomingsSinha and Poggio, Nature, 1996But humans can tell a lot about a scene from a little information…Source: “80 million tiny images” by Torralba, et al.Source: Antonio TorralbaThe goal of computer visionThe goal of computer vision•Computing the 3D shape of the worldThe goal of computer vision•Recognizing objects and peopleVision as a source of semantic informationslide credit: Fei-Fei, Fergus & TorralbaObject categorizationskybuildingflagwallbannerbuscarsbusfacestreet lampslide credit: Fei-Fei, Fergus & TorralbaThe goal of computer vision•“Enhancing” imagesThe goal of computer vision•“Enhancing” imagesTexture synthesis / increased field of view (uncropping) (image credit: Efros and Leung)Inpainting / image completion (image credit: Hays and Efros)Super-resolution / denoising (source: 2d3)The goal of computer vision•ForensicsSource: Nayar and Nishino, “Eyes for Relighting”Source: Nayar and Nishino, “Eyes for Relighting”Source: Nayar and Nishino, “Eyes for Relighting”Why study computer vision?•Millions of images being captured all the time•Lots of useful applications•The next slides show the current state of the artSource: S. LazebnikOptical character recognition (OCR)Digit recognition, AT&T labshttp://www.research.att.com/~yann/•If you have a scanner, it probably came with OCR softwareLicense plate readershttp://en.wikipedia.org/wiki/Automatic_number_plate_recognitionSource: S. SeitzAutomatic check processingSudoku grabberhttp://sudokugrab.blogspot.com/Face detection•Many new digital cameras now detect faces–Canon, Sony, Fuji, …Source: S. SeitzSmile detection?Sony Cyber-shot® T70 Digital Still Camera Source: S. SeitzObject recognition (in supermarkets)LaneHawk by EvolutionRobotics“A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “Source: S. SeitzFace recognitionWho is she? Source: S. SeitzVision-based biometrics“How the Afghan Girl was Identified by Her Iris Patterns” Read the story Source: S. SeitzLogin without a password…Fingerprint scanners on many new laptops, other devicesFace recognition systems now beginning to appear more widelyhttp://www.sensiblevision.com/Source: S. SeitzObject recognition (in mobile phones)•This is becoming real:– Microsoft Research–Point & FindSource: S. SeitziPhone Apps: (www.kooaba.com)Source: S. LazebnikThe Matrix movies, ESC Entertainment, XYZRGB, NRCSpecial effects: shape captureSource: S. SeitzPirates of the Carribean, Industrial Light and MagicSpecial effects: motion captureSource: S. SeitzSpecial effects: camera trackingBoujou, 2d3SportsSportvision first down lineNice explanation on www.howstuffworks.comSource: S. SeitzSmart cars•Mobileye–Vision systems currently in high-end BMW, GM, Volvo models –By 2010: 70% of car manufacturers.Sources: A. Shashua, S. SeitzVision-based interaction (and games)Nintendo Wii has camera-based IRtracking built in. See Lee’s work atCMU on clever tricks on using it tocreate a multi-touch display! Source: S. SeitzAssistive technologiesSony EyeToyProject Natal?Vision in spaceVision systems (JPL) used for several tasks•Panorama stitching•3D terrain modeling•Obstacle detection, position tracking•For more, read “Computer Vision on Mars” by Matthies et al.NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of