3D Computer Vision Review and Video Computing CSC I6716 Spring 2003 Midterm Review Zhigang Zhu NAC 8 203A http www cs engr ccny cuny edu zhu VisionCourse I6716 html 3D Computer Vision and Video Computing Course Outline Complete syllabus on the web pages 12 13 lectures Rough Outline 3D Computer Vision and Video Computing Part 1 Vision Basics 1 Introduction 2 Sensors 3 Image Formation and Processing hw 1 matlab 4 Features and Feature Extraction 2 lectures hw 2 Part 2 3D Vision 5 6 7 8 Camera Models Camera Calibration hw 3 Stereo Vision project assignment Visual Motion midterm exam Part 3 Video Computing 9 Video Mosaicing 10 Omnidirectional Stereo 11 Human Tracking 12 Applications Image Based Rendering Video Coding MPEG 7 etc 3D Computer Vision and Video Computing What is Computer Vision bigger picture Part 1 1 Introduction Goals Goals Approaches What Makes 3D Computer Vision Interesting Parts 2 3 Image Modeling Analysis Interpretation Interpretation is an Artificial Intelligence Problem Interpretation often goes from 2D images to 3D structures Sources of Knowledge in Vision Levels of Abstraction since we live in a 3D world Image Rendering Synthesis Composition Image Rendering is a Computer Graphics problem Rendering is from 3D model to 2D images 3D Computer Vision and Video Computing Image Processing image to image Computer Vision Image to model Computer Graphics model to image Pattern Recognition image to class Related Fields All three are interrelated image data mining video mining Artificial Intelligence machine smarts Photogrammetry camera geometry 3D reconstruction Applications Medical Imaging CAT MRI 3D reconstruction 2nd meaning Video Coding encoding decoding compression transmission AI Physics basics Mathematics basics Neuroscience wetware to concept Computer Science programming tools and skills basics 3D Computer Vision and Video Computing Visual Inspection Robotics Intelligent Image Tools Image Compression MPEG 1 2 4 7 Document Analysis OCR Image Libraries DL Virtual Environment Construction Environment Media and Entertainment Medicine Astronomy Law Enforcement surveillance security Traffic and Transportation Tele Conferencing and e Learning Applications 3D Computer Vision and Video Computing Static monocular reflectance data Motion sequences camcorders Stereo 2 cameras Range data Range finder Non visual sensory data Film Video cameras Digital cameras infrared IR ultraviolet UV microwaves Many more 2 Sensors 3D Computer Vision and Video Computing The Electromagnetic Spectrum Visible Spectrum 700 nm 400 nm 3D Computer Vision and Video Computing Light and Optics 3 Image Formations Pinhole camera model Perspective projection Thin lens model Fundamental equation Distortion spherical chromatic aberration radial distortion option Reflection and Illumination color lambertian and specular surfaces Phong BDRF option Sensing Light Conversion to Digital Images Sampling Theorem Other Sensors frequency type 3D Computer Vision and Video Computing Image Enhancement Brightness mapping Contrast stretching enhancement Histogram modification Noise Reduction Mathematical Techniques 4 Feature Extraction Convolution Gaussian Filtering Edge and Line Detection and Extraction Region Segmentation Contour Extraction Corner Detection 3D Computer Vision and Video Computing Edgels Define a local edge or edgel to be a rapid change in the image function over a small area implies that edgels should be detectable over a local neighborhood Edgels are NOT contours boundaries or lines edgels may lend support to the existence of those structures these structures are typically constructed from edgels Edgels have properties Orientation Magnitude Length typically a unit length 3D Computer Vision and Video Computing First order edge detectors lecture required Mathematics 1x2 Roberts Sobel Prewitt Canny edge detector after class reading Second order edge detector after class reading Edge Detection Laplacian LOG DOG Hough Transform detect by voting Lines Circles Other shapes 3D Computer Vision and Video Computing Edge Detection Typical Noise Smoothing Suppress as much noise as possible while retaining true edges In the absence of other information assume white noise with a Gaussian distribution Edge Enhancement Design a filter that responds to edges filter output high are edge pixels and low elsewhere Edge Localization Determine which edge pixels should be discarded as noise and which should be retained thin wide edges to 1 pixel width nonmaximum suppression establish minimum value to declare a local maximum from edge filter to be an edge thresholding 3D Computer Vision and Video Computing Closely Related Disciplines Image processing image to mage Pattern recognition image to classes Photogrammetry obtaining accurate measurements from images What is 3 D three dimensional Vision Part 2 3D Vision Motivation making computers see the 3D world as humans do Computer Vision 2D images to 3D structure Applications robotics VR Image based rendering 3D video Lectures on 3 D Vision Fundamentals Part 2 Camera Geometric Model 2 this class topic 5 Camera Calibration 2 topic 6 Stereo 2 topic 7 Motion 2 topic 8 3D Computer Vision and Video Computing Geometric Projection of a Camera Pinhole camera model Perspective projection Weak Perspective Projection Camera Parameters Intrinsic Parameters define mapping from 3D to 2D Extrinsic parameters define viewpoint and viewing direction Basic Vector and Matrix Operations Rotation Camera Models Revisited Linear Version of the Projection Transformation Equation 5 Camera Models Perspective Camera Model Weak Perspective Camera Model Affine Camera Model Camera Model for Planes Summary 3D Computer Vision and Video Computing Calibration Find the intrinsic and extrinsic parameters Basic equations from Lecture 5 Estimating the Image center using vanishing points Orthocenter Theorem SVD Singular Value Decomposition and Homogeneous System Focal length Aspect ratio and extrinsic parameters Discussion Why not do all the parameters together Projection Matrix Approach after class reading Problem and assumptions Direct parameter estimation approach Projection matrix approach Direct Parameter Estimation Approach 6 Camera Calibration Estimating the projection matrix M Computing the camera parameters from M Discussion Comparison and Summary Any difference 3D Computer Vision and Video Computing Problem Infer 3D structure of a scene from two or more images taken from different viewpoints Two primary Sub