1CSE486, Penn StateRobert CollinsLecture 19:Essential and Fundamental MatricesCSE486, Penn StateRobert CollinsEpipolar Geometryimage1image 2Epipole : location of cam2as seen by cam1.Epipole : location of cam1as seen by cam2.CSE486, Penn StateRobert CollinsEpipolar Geometryimage1image 2Corresponding pointslie on conjugate epipolar linesCSE486, Penn StateRobert CollinsThis Lecture…image1image 2Given a point in one image, how do we determinethe corresponding epipolar line to search along inthe second image?CSE486, Penn StateRobert CollinsEssential MatrixThe essential and fundamental matrices are 3x3 matricesthat “encode” the epipolar geometry of two views.Motivation: Given a point in one image, multiplyingby the essential/fundamental matrix will tell us which epipolar line to search along in the second view.CSE486, Penn StateRobert CollinsEssential MatrixeerreellOOrrpprrPPppllOOllTTPPllPPrrR,T = rotation, and translationS=E=RS is “essential matrix”2CSE486, Penn StateRobert CollinsEssential Matrix Properties• has rank 2  has both a left and right nullspace (important!!!!)• depends only on the EXTRINSIC Parameters (R & T)CSE486, Penn StateRobert CollinsLonguet-Higgins equationCSE486, Penn StateRobert CollinsLonguet-Higgins equationImportance of Longuet-Higgens ...This relates viewing raysThis relates 2D film pointsCSE486, Penn StateRobert CollinsLonguet-Higgins Makes Sense• Note, there is nothing magic about Longuet-Higgins equation.• A film point can also be thought of as aviewing ray. They are equivalent.• (u,v) 2D film point• (u,v,f) 3D point on film plane• k(u,v,f) viewing ray into the scene• k(X, Y, Z) ray through point P in the scene [hint: k=f/Z, and we have u=fX/Z, v=fY/Z].CSE486, Penn StateRobert CollinsEpipolar Lines• Let l be a line in the image:• Using homogeneous coordinates:CSE486, Penn StateRobert CollinsEpipolar Lines• Remember:pprr belongs to belongs to epipolarepipolar line in the right image defined by line in the right image defined by3CSE486, Penn StateRobert CollinsEpipolar Lines• Remember:ppll belongs to belongs to epipolarepipolar line in the left image defined by line in the left image defined by CSE486, Penn StateRobert CollinsEpipoles• Remember: epipoles belong to the epipolar lines• And they belong to all the epipolar linesWe can use this to compute the location of the epipoles.There will be an example, shortly...CSE486, Penn StateRobert CollinsEssential Matrix SummaryLonguet-Higgins equationEpipolar lines:Epipoles:CSE486, Penn StateRobert CollinsFundamental MatrixThe essential matrix uses CAMERA coordinatesTo use image coordinates we must consider theTo use image coordinates we must consider theINTRINSIC camera parameters:INTRINSIC camera parameters:Pixel coord(row,col)Affine transform matrixCamera (film) coordCSE486, Penn StateRobert CollinsFundamental Matrixshort version: The same equation works inpixel coordinates too!CSE486, Penn StateRobert CollinsFundamental Matrix Properties• has rank 2• depends on the INTRINSIC and EXTRINSIC Parameters (f, etc ;R & T)Analogous to essential matrix. The fundamentalmatrix also tells how pixels (points) in each image are related to epipolar lines in the other image.4CSE486, Penn StateRobert CollinsExampleCSE486, Penn StateRobert CollinsExampleCSE486, Penn StateRobert CollinsExample -0.00310695 -0.0025646 2.96584 -0.028094 -0.00771621 56.3813 13.1905 -29.2007 -9999.79F = CSE486, Penn StateRobert CollinsExample -0.00310695 -0.0025646 2.96584 -0.028094 -0.00771621 56.3813 13.1905 -29.2007 -9999.79F = x = 343.5300 y = 221.7005left343.53221.70 1.0 0.0001 0.0045 -1.1942 0.0295 0.9996 -265.1531normalize so sum of squaresof first two terms is 1 (optional)CSE486, Penn StateRobert CollinsExample -0.00310695 -0.0025646 2.96584 -0.028094 -0.00771621 56.3813 13.1905 -29.2007 -9999.79F = x = 343.5300 y = 221.7005left343.53221.70 1.0 0.0295 0.9996 -265.1531rightCSE486, Penn StateRobert CollinsExample -0.00310695 -0.0025646 2.96584 -0.028094 -0.00771621 56.3813 13.1905 -29.2007 -9999.79x = 205.5526 y = 80.5000right( 205.5526 80.5 1.0)L = (0.0010 -0.0030 -0.4851) (0.3211 -0.9470 -151.39)5CSE486, Penn StateRobert CollinsExample -0.00310695 -0.0025646 2.96584 -0.028094 -0.00771621 56.3813 13.1905 -29.2007 -9999.79x = 205.5526 y = 80.5000right( 205.5526 80.5 1.0)L= (0.3211 -0.9470 -151.39)leftCSE486, Penn StateRobert CollinsExamplewhere is the epipole?F * eL = 0vector in the rightnullspace of matrix FHowever, due to noise,F may not be singular.So instead, next bestthing is eigenvectorassociated with smallesteigenvalue of FCSE486, Penn StateRobert CollinsExample>> [u,d] = eigs(F’ * F)u = -0.0013 0.2586 -0.9660 0.0029 -0.9660 -0.2586 1.0000 0.0032 -0.0005d = 1.0e8* -1.0000 0 0 0 -0.0000 0 0 0 -0.0000eigenvector associated with smallest eigenvalue>> uu = u(:,3)uu = ( -0.9660 -0.2586 -0.0005)>> uu / uu(3) : to get pixel coords(1861.02 498.21 1.0)CSE486, Penn StateRobert CollinsExamplewhere is the epipole?e’r * F = 0 F’ * er = 0vector in the rightnullspace of matrix F’However, due to noise,F’ may not be singular.So instead, next bestthing is eigenvectorassociated with smallesteigenvalue of F’CSE486, Penn StateRobert CollinsExample>> [u,d] = eigs(F * F’)u = -0.0003 -0.0618 -0.9981 -0.0056 -0.9981 0.0618 1.0000 -0.0056 0.0001d = 1.0e8* -1.0000 0 0 0 -0.0000 0 0 0 -0.0000eigenvector associated with smallest eigenvalue>> uu = u(:,3)uu = (-0.9981 0.0618 0.0001)>> uu / uu(3) : to get pixel coords(-19021.8 1177.97