Epipolar Geometry class 11Multiple View Geometry course schedule (subject to change)More Single-View GeometrySlide Number 5Slide Number 6Slide Number 7Slide Number 8Slide Number 9Slide Number 10Slide Number 11Slide Number 12Slide Number 13Slide Number 14Slide Number 15Slide Number 16Slide Number 17Slide Number 18Slide Number 19Epipolar Line HomographySlide Number 21Pure Translation camera motionSlide Number 23Slide Number 24Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Next class: 3D reconstructionEpipolar Geometry class 11Multiple View GeometryComp 290-089Marc PollefeysMultiple View Geometry course schedule (subject to change)Jan. 7, 9 Intro & motivation Projective 2D GeometryJan. 14, 16 (no class) Projective 2D GeometryJan. 21, 23 Projective 3D Geometry (no class)Jan. 28, 30 Parameter Estimation Parameter EstimationFeb. 4, 6 Algorithm Evaluation Camera ModelsFeb. 11, 13 Camera Calibration Single View GeometryFeb. 18, 20 Epipolar Geometry3D reconstructionFeb. 25, 27 Fund. Matrix Comp. Structure Comp.Mar. 4, 6 Planes & Homographies Trifocal TensorMar. 18, 20 Three View Reconstruction Multiple View GeometryMar. 25, 27 MultipleView Reconstruction Bundle adjustmentApr. 1, 3 Auto-Calibration PapersApr. 8, 10 Dynamic SfM PapersApr. 15, 17 Cheirality PapersApr. 22, 24 Duality Project DemosMore Single-View Geometry• Projective cameras and planes, lines, conics and quadrics.• Camera calibration and vanishing points, calibrating conic and the IAC**CPPQ =TconeQCPP =TlPT=ΠTwo-view geometryEpipolar geometry3D reconstructionF-matrix comp.Structure comp.(i) Correspondence geometry: Given an image point x in the first view, how does this constrain the position of the corresponding point x’ in the second image?(ii) Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i }, i=1,…,n, what are the cameras P and P’ for the two views?(iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?Three questions:The epipolar geometryC,C’,x,x’ and X are coplanar(a)The epipolar geometryWhat if only C,C’,x are known?bThe epipolar geometryAll points on π project on l and l’aThe epipolar geometryFamily of planes π and lines l and l’Intersection in e and e’bThe epipolar geometryepipoles e,e’= intersection of baseline with image plane = projection of projection center in other image= vanishing point of camera motion directionan epipolar plane = plane containing baseline (1-D family)an epipolar line = intersection of epipolar plane with image(always come in corresponding pairs)Example: converging camerasExample: motion parallel with image planeThe fundamental matrix Falgebraic representation of epipolar geometry l'x awe will see that mapping is (singular) correlation (i.e. projective mapping from points to lines) represented by the fundamental matrix FThe fundamental matrix Fgeometric derivationxHx'π=x'e'l'×=[]FxxHe'π==×mapping from 2-D to 1-D family (rank 2)The fundamental matrix Falgebraic derivation()λCxPλX +=+()IPP =+[]+×= PP'e'FxPP'CP'l+×=(note: doesn’t work for C=C’ ⇒ F=0)xP+()λXThe fundamental matrix Fcorrespondence condition0Fxx'T=The fundamental matrix satisfies the condition that for any pair of corresponding points x↔x’in the two images()0l'x'T=The fundamental matrix FF is the unique 3x3 rank 2 matrix that satisfies x’TFx=0 for all x↔x’(i) Transpose: if F is fundamental matrix for (P,P’), then FT is fundamental matrix for (P’,P)(ii) Epipolar lines: l’=Fx & l=FTx’(iii) Epipoles: on all epipolar lines, thus e’TFx=0, ∀x ⇒e’TF=0, similarly Fe=0(iv) F has 7 d.o.f. , i.e. 3x3-1(homogeneous)-1(rank2)(v) F is a correlation, projective mapping from a point x to a line l’=Fx (not a proper correlation, i.e. not invertible)Epipolar Line HomographyThe epipolar line geometryl,l’ epipolar lines, k line not through e⇒ l’=F[k]x l and symmetrically l=FT[k’]x l’lk×ekllFk×e'(pick k=e, since eTe≠0)[]leFl'×=[]l'e'FlT×=Pure Translation camera motionFundamental matrix for pure translationForward motionFundamental matrix for pure translation[][]×∞×==e'He'F()RKKH1−∞=×⎥⎥⎦⎤⎢⎢⎣⎡=0101-00000F()T1,0,0e'=example:y'y =⇔= 0Fxx'T0]X|K[IPXx==⎥⎦⎤⎢⎣⎡==ZxKt]|K[IXP'x'-1ZKt/xx'+=ZX,Y,Z x/K)(-1T=motion starts at x and moves towards e, faster depending on Zpure translation: F only 2 d.o.f., xT[e]x x=0 ⇒ auto-epipolarGeneral motionZt/K'xRKK'x'-1+=[]0Hxe''x =×T[]0xˆe''x =×TGeometric representation of F()2/FFFTS+=()2/FFFTA−=()ASFFF+=0FxxT=xx↔()0xFxAT≡0xFxST=Fs : Steiner conic, 5 d.o.f.Fa =[xa ]x : pole of line ee’ w.r.t. Fs , 2 d.o.f.Pure planar motion Steiner conic Fs is degenerate (two lines)Projective transformation and invariance-1-TFHH'Fˆ x'H''xˆ Hx,xˆ=⇒==Derivation based purely on projective conceptsF invariant to transformations of projective 3-space()FP'P, a()P'P,F auniquenot uniquecanonical formm]|[MP'0]|[IP==[]MmF×=Projective ambiguity of cameras given Fprevious slide: at least projective ambiguitythis slide: not more! Show that if F is same for (P,P’) and (P,P’), there exists a projective transformation H so that P=HP and P’=HP’~ ~~~ ]a~|A~['P~ 0]|[IP~ a]|[AP' 0]|[IP ====[][]A~a~AaF××==()T1avAA~ kaa~+==−klemma:Canonical cameras given FF matrix corresponds to P,P’ iff P’TFP is skew-symmetric()X0,FPXP'XTT∀=Possible choice: ]e'|F][[e'P' 0]|[IP×==The essential matrix~fundamental matrix for calibrated cameras (remove K)[]××== t]R[RRtET0xˆE'xˆT=FKK'ET=() x'K'xˆ x;Kxˆ-1-1==5 d.o.f. (3 for R; 2 for t up to scale)E is essential matrix if and only iftwo singularvalues are equal (and third=0)T0)VUdiag(1,1,E =Four possible reconstructions from E(only one solution where points is in front of both cameras)Next class: 3D