13D Computer Visionand Video Computing3D Vision3D VisionTopic 1 of Part IICamera ModelsCSC I6716Spring2011Zhigang Zhu, City College of New York [email protected] Computer Visionand Video Computing3D Vision3D Vision Closely Related Disciplines  Image Processing – images to mages Computer Graphics – models to images Computer Vision – images to models Photogrammetry – obtaining accurate measurements from imagesWhat is 3-D ( three dimensional) Vision? Motivation: making computers see (the 3D world as humans do) Computer Vision: 2D images to 3D structure Applications : robotics / VR /Image-based rendering/ 3D videoLectures on 3-D Vision Fundamentals Camera Geometric Models (1 lecture) Camera Calibration (2 lectures) Stereo (2 lectures) Motion (2 lectures)23D Computer Visionand Video ComputingLecture OutlineLecture Outline Geometric Projection of a Camera Pinhole camera model Perspective projection Weak-Perspective ProjectionCamera Parameters Intrinsic Parameters: define mapping from 3D to 2D Extrinsic parameters: define viewpoint and viewing direction Basic Vector and Matrix Operations, Rotation Camera Models Revisited Linear Version of the Projection Transformation Equation Perspective Camera Model Weak-Perspective Camera Model Affine Camera Model Camera Model for Planes Summary3D Computer Visionand Video ComputingLecture AssumptionsLecture Assumptions Camera Geometric Models Knowledge about 2D and 3D geometric transformations Linear algebra (vector, matrix) This lecture is only about geometry GoalBuild up relation between 2D images and 3D scenes-3D Graphics (rendering): from 3D to 2D-3D Vision (stereo and motion): from 2D to 3D-Calibration: Determning the parameters for mapping33D Computer Visionand Video ComputingImage FormationImage Formation3D Computer Visionand Video ComputingImage FormationImage FormationLight (Energy) SourceSurfacePinhole LensImaging PlaneWorld Optics Sensor SignalCamera: Spec & Pose3D Scene2D Image43D Computer Visionand Video ComputingPinhole Camera ModelPinhole Camera Model Pin-hole is the basis for most graphics and vision Derived from physical construction of early cameras Mathematics is very straightforward 3D World projected to 2D Image Image inverted, size reduced Image is a 2D plane: No direct depth information Perspective projection  f called the focal length of the lens given image size, change f will change FOV and figure sizes3D Computer Visionand Video ComputingFocal Length, FOVFocal Length, FOV Consider case with object on the optical axis: Optical axis: the direction of imaging Image plane: a plane perpendicular to the optical axis Center of Projection (pinhole), focal point, viewpoint, nodal point Focal length: distance from focal point to the image plane FOV : Field of View – viewing angles in horizontal and vertical directionsfzviewpointImage plane53D Computer Visionand Video ComputingFocal Length, FOVFocal Length, FOV Consider case with object on the optical axis:fzOut of viewImage plane Optical axis: the direction of imaging Image plane: a plane perpendicular to the optical axis Center of Projection (pinhole), focal point, viewpoint, , nodal point Focal length: distance from focal point to the image plane FOV : Field of View – viewing angles in horizontal and vertical directions Increasing f will enlarge figures, but decrease FOV3D Computer Visionand Video ComputingEquivalent GeometryEquivalent Geometry Consider case with object on the optical axis:fz More convenient with upright image:fzProjection plane z = f Equivalent mathematically63D Computer Visionand Video ComputingPerspective ProjectionPerspective Projection Compute the image coordinates of p in terms of the world (camera) coordinates of P. Origin of camera at center of projection Z axis along optical axis Image Plane at Z = f; x // X and y//YxyZP(X,Y,Z)p(x, y)Z = f0YX3D Computer Visionand Video ComputingReverse ProjectionReverse Projection Given a center of projection and image coordinates of a point, it is not possible to recover the 3D depth of the point from a single image.In general, at least two images of the same point taken from two different locations are required to recover depth.73D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdam : what do you see in this picture?straight linesizeparallelism/angleshapeshape of planes depth3D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdamstraight linesizeparallelism/angleshapeshape of planes depth83D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdamstraight linesizeparallelism/angleshapeshape of planes depth3D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdamstraight linesizeparallelism/angleshapeshape of planes depth93D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdamstraight linesizeparallelism/angleshapeshape of planes depth3D Computer Visionand Video ComputingPinhole camera imagePinhole camera imagePhoto by Robert Kosara, [email protected]://www.kosara.net/gallery/pinholeamsterdam/pic01.html Amsterdamstraight linesizeparallelism/angleshapeshape of planes  parallel to imagedepth103D Computer Visionand Video ComputingPinhole camera imagePinhole camera image- We see spatial shapes rather than individual pixels- Knowledge: top-down vision belongs to human- Stereo &Motion most successful in 3D CV & application- You can see it but you don't know how…Amsterdam: what do you see?straight linesizeparallelism/angleshapeshape of planes  parallel to imageDepth ? stereo motionsizestructure …3D Computer Visionand Video ComputingYet