1CSE486, Penn StateRobert CollinsLecture 12: Camera ProjectionReading: T&V Section 2.4CSE486, Penn StateRobert CollinsImaging GeometryVUWObject of Interestin World CoordinateSystem (U,V,W)CSE486, Penn StateRobert CollinsImaging GeometryZfCamera Coordinate System (X,Y,Z). • Z is optic axis• Image plane located f unitsout along optic axis• f is called focal lengthXYCSE486, Penn StateRobert CollinsImaging GeometryVUWZForward Projection onto image plane.3D (X,Y,Z) projected to 2D (x,y)yxXYCSE486, Penn StateRobert CollinsImaging GeometryVUWZyOur image gets digitizedinto pixel coordinates (u,v)xXYuvCSE486, Penn StateRobert CollinsImaging GeometryVUWZyWorld CoordinatesCameraCoordinatesImage (film)CoordinatesPixelCoordinatesuvxXY2CSE486, Penn StateRobert CollinsForward ProjectionUVWXYZxyuvWorldCoordsCameraCoordsFilmCoordsPixelCoordsWe want a mathematical model to describehow 3D World points get projected into 2DPixel coordinates.Our goal: describe this sequence of transformations by a big matrix equation!CSE486, Penn StateRobert CollinsBackward ProjectionUVWXYZxyuvWorldCoordsCameraCoordsFilmCoordsPixelCoordsNote, much of vision concerns trying toderive backward projection equations torecover 3D scene structure from images (via stereo or motion)But first, we have to understand forward projection…CSE486, Penn StateRobert CollinsForward ProjectionUVWXYZxyuvWorldCoordsCameraCoordsFilmCoordsPixelCoords3D-to-2D Projection• perspective projectionWe will start here in the middle, since we’ve alreadytalked about this when discussing stereo.CSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfxO.Camps, PSUZP =P =(X,Y,Z)yScene PointImage PointPerspective Projection EqnsYXxCSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfxO.Camps, PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYXxZfderived via similar triangles ruleCSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfxO.Camps, PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYXxZfZYyfderived via similar triangles rule3CSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfxO.Camps, PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYSo how do we represent this as a matrix equation? We need to introduce homogeneous coordinates.CSE486, Penn StateRobert CollinsHomogeneous CoordinatesRepresent a 2D point (x,y) by a 3D point (x’,y’,z’) byadding a “fictitious” third coordinate.By convention, we specify that given (x’,y’,z’) we canrecover the 2D point (x,y) as'''' zyyzxxNote: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero k (can be negative as well as positive)CSE486, Penn StateRobert CollinsPerspective Matrix Equation(in Camera Coordinates) 10001000000'''ZYXffzyxZYfyZXfxCSE486, Penn StateRobert CollinsForward ProjectionUVWXYZxyuvWorldCoordsCameraCoordsFilmCoordsPixelCoordsRigid Transformation (rotation+translation)between world and camera coordinate systemsCSE486, Penn StateRobert CollinsWorld to Camera TransformationXXYYZZPCUUVVWWPWAvoid confusion: Pw and Pc are not two different points. They are the same physical point, described in two different coordinate systems.CSE486, Penn StateRobert CollinsWorld to Camera TransformationXXYYZZPCUUVVWWPWTranslate by - C (align origins)CRotate toalign axesRPC= R ( PW- C )4CSE486, Penn StateRobert CollinsMatrix Form, Homogeneous CoordsPC= R ( PW- C )1000100100zyxccc0011WVU1000r13r12r11r23r22r21r33r32r311ZYXCSE486, Penn StateRobert CollinsExample: Simple Stereo SystemXYZlocated atlocated at(0,0,0)(0,0,0)leftcamerazzxxyy(X,Y,Z)TTxxrightcameralocated atlocated at(T(Txx,0,0),0,0)zzxxyy( , )( , )Left camera located at world origin (0,0,0)and camera axes aligned with world coord axes.CSE486, Penn StateRobert Collins1000100100zyxccc0011000r13r12r11r23r22r21r33r32r311ZYX1WVUSimple Stereo, Left Camera1 0 00 1 00 0 1camera axes alignedwith world axes000located at world position (0,0,0)=CSE486, Penn StateRobert CollinsSimple Stereo Projection EquationsLeft cameraCSE486, Penn StateRobert CollinsExample: Simple Stereo SystemXYZlocated atlocated at(0,0,0)(0,0,0)leftcamerazzxxyy(X,Y,Z)TTxxrightcameralocated atlocated at(T(Txx,0,0),0,0)zzxxyy( , )( , )Right camera located at world location (Tx,0,0)and camera axes aligned with world coord axes.CSE486, Penn StateRobert Collins1000100100zyxccc0011000r13r12r11r23r22r21r33r32r311ZYX1WVUSimple Stereo, Right Camera1 0 00 1 00 0 1camera axes alignedwith world axes-Tx00located at world position (Tx,0,0)=5CSE486, Penn StateRobert CollinsRight cameraSimple Stereo Projection EquationsLeft cameraCSE486, Penn StateRobert CollinsBob’s sure-fire way(s) tofigure out the rotation1000100100zyxccc0011WVU1000r13r12r11r23r22r21r33r32r311ZYXPC= R PWforget about thiswhile thinkingabout rotationsThis equation says how vectors in the world coordinate system (including the coordinate axes) get transformed into the camera coordinate system.CSE486, Penn StateRobert CollinsFiguring out Rotations1WVU1000r13r12r11r23r22r21r33r32r311ZYXPC= R PWwhat if world x axis (1,0,0) corresponds to camera axis (a,b,c)?1WVU1000r13r12r11r23r22r21r33r32r311ZYX100abc1WVU1000r13r12r11r23r22r21r33r32r311ZYX100abcabcwe can immediately write down the first column of R!CSE486, Penn StateRobert CollinsFiguring out Rotations1WVU1000r13r12r11r23r22r21r33r32r311ZYXand likewise with world Y axis and world Z axis...axis is world coordssame axis in camera coordsworld X axis (1,0,0)in camera coordsworld Y axis (0,1,0)in camera coordsworld Z axis (0,0,1)in camera coordsCSE486, Penn StateRobert CollinsFiguring out Rotations1WVU1000r31r21r11r32r22r12r33r23r131ZYXAlternative approach: sometimes it is easier to specifywhat camera X,Y,or Z axis is in world coordinates. Thendo rearrange the equation as follows.PC= R PWR-1PC= PWRTPC= PWCSE486, Penn StateRobert CollinsFiguring out Rotationswhat if camera X axis (1,0,0) corresponds to world axis (a,b,c)?1WVU1000r31r21r11r32r22r12r33r23r131ZYXRTPC= PW1WVU1000r31r21r11r32r22r12r33r23r131ZYX100abcwe can