Visual Simulation CAP 6938View Morphing (Seitz & Dyer, SIGGRAPH 96)But First: Multi-View Projective GeometryMulti-View Projective GeometryEpipolar GeometryTransfer from Epipolar LinesEpipolar AlgebraSimplifying: p’T T(Rp) = 0Linear Multiview RelationsThe Trifocal TensorSlide 11Uniqueness ResultSlide 13Image MorphingImage Morphing for View Synthesis?Special Case: Parallel CamerasUncalibrated PrewarpingPowerPoint PresentationSlide 19Slide 20Slide 21Slide 22View Morphing SummaryVisual SimulationCAP 6938Dr. Hassan Foroosh Dept. of Computer ScienceUCF© Copyright Hassan Foroosh 2002Morphed Morphed ViewViewVirtual CameraVirtual CameraView Morphing (Seitz & Dyer, SIGGRAPH 96)View interpolation (ala McMillan) butno depthno camera informationPhotograPhotographphPhotograPhotographphBut First: Multi-View Projective GeometryLast time (single view geometry)Vanishing PointsPoints at InfinityVanishing LinesThe Cross-RatioToday (multi-view geometry)Point-line dualityEpipolar geometryThe Fundamental MatrixAll quantities on these slides are in homogeneous coordinates except when specified otherwiseMulti-View Projective GeometryHow to relate point positions in different views?Central question in image-based renderingProjective geometry gives us some powerful toolsconstraints between two or more imagesequations to transfer points from one image to anotherscene pointfocal pointimage planeEpipolar GeometryWhat does one view tell us about another?Point positions in 2nd view must lie along a known lineEpipolar ConstraintExtremely useful for stereo matchingReduces problem to 1D search along conjugate epipolar linesAlso useful for view interpolation...epipolar lineepipolar lineepipolar planeepipolesTransfer from Epipolar LinesWhat does one view tell us about another?Point positions in 2nd view must lie along a known lineTwo views determines point position in a third imageBut doesn’t work if point is in the trifocal plane spanned by all three camerasbad case: three cameras are colinearinput image input imageoutput imageEpipolar AlgebraHow do we compute epipolar lines?Can trace out lines, reproject. But that is overkillYpp’TRXZXYZNote that p’ is  to Tp’So 0 = p’T Tp = p’T T(Rp + T) = p’T T(Rp) p’ = Rp + TSimplifying: p’T T(Rp) = 0We can write a cross-product ab as a matrix equationa  b = Ab where000xyxzyzzyxTherefore: Where E = TR is the 3x3 “essential matrix”Holds whenever p and p’ correspond to the same scene pointEpp'T0Properties of EEp is the epipolar line of p; p’T E is the epipolar line of p’p’T E p = 0 for every pair of corresponding points0 = Ee = e’T E where e and e’ are the epipolesE has rank < 3, has 5 independent parametersE tells us everything about the epipolar geometryLinear Multiview RelationsThe Essential Matrix: 0 = p’T E pFirst derived by Longuet-Higgins, Nature 1981also showed how to compute camera R and T matrices from EE has only 5 free parameters (three rotation angles, two transl. directions)Only applies when cameras have same internal parameterssame focal length, aspect ratio, and image centerThe Fundamental Matrix: 0 = p’T F pF = (A’-1)T E A-1, where A3x3 and A’3x3 contain the internal parametersGives epipoles, epipolar linesF (like E) defined only up to a scale factor & has rank 2 (7 free params)Generalization of the essential matrixCan’t uniquely solve for R and T (or A and A’) from FCan be computed using linear methods R. Hartley, In Defence of the 8-point Algorithm, ICCV 95Or nonlinear methods: Xu & Zhang, Epipolar Geometry in Stereo, 1996The Trifocal TensorWhat if you have three views?Can compute 3 pairwise fundamental matricesHowever there are more constraintsit should be possible to resolve the trifocal problem Answer: the trifocal tensorintroduced by Shashua, Hartley in 1994/1995a 3x3x3 matrix T (27 parameters)gives all constraints between 3 viewscan use to generate new views without trifocal probs. [Shai & Avidan]linearly computable from point correspondencesHow about four views? five views? N views?There is a quadrifocal tensor [Faugeras & Morrain, Triggs, 1995]But: all the constraints are expressed in the trifocal tensors, obtained by considering every subset of 3 camerasMorphed Morphed ViewViewVirtual CameraVirtual CameraView Morphing (Seitz & Dyer, SIGGRAPH 96)View interpolation (ala McMillan) butno depthno camera informationPhotograPhotographphPhotograPhotographphUniqueness ResultGivenAny two images of a Lambertian sceneNo occlusionsResult: all views along C1C2 are uniquely determinedView Synthesis is solvable whenCameras are uncalibratedDense pixel correspondence is not availableShape reconstruction is impossibleUniqueness ResultRelies on Monotonicity AssumptionLeft-to-right ordering of points is the same in both imagesused often to facilitate stereo matchingImplies no occlusions on line between C1 and C2Image MorphingLinear Interpolation of 2D shape and colorPhotograPhotographphPhotograPhotographphPhotograPhotographphPhotograPhotographphMorphed Morphed ImageImageImage Morphing for View Synthesis?We want to high quality view interpolationsCan image morphing do this?Goal: extend to handle changes in viewpointProduce valid camera transitionsMorphing parallel views new parallel viewsProjection matrices have a special formthird rows of projection matrices are equalLinear image motion linear camera motionSpecial Case: Parallel CamerasMorphedMorphed ImageImageRightRight ImageImageLeftLeft ImageImageUncalibrated PrewarpingParallel cameras have a special epipolar geometryEpipolar lines are horizontalCorresponding points have the same y coordinate in both imagesWhat fundamental matrix does this correspond to?010100000ˆFPrewarp procedure:Compute F matrix given 8 or more correspondencesCompute homographies H and H’ such thateach homography composes two rotations, a scale, and a translationTransform first image by H-1, second image by H’-1FFHH'ˆT1. Prewarp align views 2. Morph  move camera1. Prewarp align