3D Urban ModelingSlide 23D Urban Modeling course schedule (tentative)Course materialGeometry primerSlide 6Geometry primer Hierarchy of 2D transformationsGeometry primer Hierarchy of 3D transformationsCamera modelProjector modelSlide 11Slide 12Slide 13Slide 14Slide 15Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Radial distortionSlide 24Meydenbauer cameraSlide 26Slide 27Slide 28Slide 29Slide 30Singular Value DecompositionSlide 32Slide 33Slide 34Slide 35Gold Standard algorithmSlide 37Slide 38Slide 39Slide 40Slide 41Image of absolute conicSlide 43Slide 44Self-calibration using absolute conicPractical linear self-calibration3D Urban ModelingMarc PollefeysJan-Michael Frahm, Philippos MordohaiFall 2006 / Comp 790-089Tue & Thu 14:00-15:153D Urban Modeling•Basics: sensor, techniques, …•Video, LIDAR, GPS, IMU, …•SfM, Stereo, …•Robot Mapping, GIS, …•Review state-of-the-art•Video and LIDAR-based systems•Projects to experiment•Virtual UNC-Chapel Hill•DARPA Urban Challenge•…3D Urban Modeling course schedule(tentative)Sep. 5, 7 IntroductionVideo-based Urban 3D CaptureSep. 12, 14 CamerasSep. 19, 21Sep. 26, 28 Project proposalsOct. 3, 5Oct. 10, 12Oct. 17, 19 (fall break)Oct. 24, 26 Project Status UpdateOct. 31, Nov. 2Nov. 7, 9Nov. 14, 16Nov. 21, 23 (Thanksgiving)Nov. 28, 30Dec. 5 Project demonstrations (classes ended)Note: Dec. 3 is CVPR deadlineCourse materialSlides, notes, papers and referencessee course webpage/wiki (later)On-line “shape-from-video” tutorial:http://www.cs.unc.edu/~marc/tutorial.pdfhttp://www.cs.unc.edu/~marc/tutorial/Geometry primerl'lx Incidence of point and line x'xl 0xl T0Xπ T0ll*CT1* CC0QXX T0πQπ*T-1*QQ JoinsConics and dual conics0xx CTIncidence of point and plane JoinsConics and dual conics0XπππT3T2T10πXXXT3T2T1Geometry primerTransformation for linesll'-THTransformation for conics-1-TCHHC 'Transformation for dual conicsTHHCC**' xx' HTransformation for pointsXX' Hππ'-TH-1-TQHHQ'THHQQ'**Transformation for planesTransformation for quadricsTransformation for dual quadricsTransformation for pointsGeometry primer Hierarchy of 2D transformations10022211211yxtaataa10022211211yxtsrsrtsrsr333231232221131211hhhhhhhhh10022211211yxtrrtrrProjective8dofAffine6dofSimilarity4dofEuclidean3dofConcurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratioParallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞Ratios of lengths, angles.The circular points I,Jlengths, areas.invariantstransformed squaresGeometry primer Hierarchy of 3D transformationsvTvtAProjective15dofAffine12dofSimilarity7dofEuclidean6dofIntersection and tangencyParallellism of planes,Volume ratios, centroids,The plane at infinity π∞Angles, ratios of lengthThe absolute conic Ω∞Volume10tAT10tRTs10tRTCamera modelRelation between pixels and rays in space?Projector modelRelation between pixels and rays in space(dual of camera)(main geometric difference is vertical principal point offset to reduce keystone effect)?Affine camerasPinhole camera modelTTZfYZfXZYX )/,/(),,( 101001ZYXffZfYfXZYXlinear projection in homogeneous coordinates!homogeneous coordinatesnon-homogeneous coordinatesPinhole camera model10100ZYXffZfYfX10101011ZYXffZfYfXPXx  0|I)1,,(diagP ffPrincipal point offsetTyxTpZfYpZfXZYX )/,/(),,( principal pointTyxpp ),(101001ZYXpfpfZZpfYZpfXZYXyxxxPrincipal point offset10100ZYXpfpfZZpfYZpfXyxxx camX0|IKx 1yxpfpfKcalibration matrixObject motionCamera motionCCD camera1yyxxppK11yxyxpfpfmmKGeneral projective camera1yxxxppsK1yxxxppK t|IKRP non-singular11 dof (5+3+3) t|RKP intrinsic camera parametersextrinsic camera parametersCamera matrix decompositionFinding the camera center0PC (use SVD to find null-space)Finding the camera orientation and internal parametersKR(use RQ decomposition ~QR)QR=( )-1= -1 -1QR(if only QR, invert)PXλC)P(Xx (for all X and λ  C must be camera center) t|RKP Affine camerasRadial distortion•Due to spherical lenses (cheap)•Model:RyxyxKyxKyx ...))()(1(),(2222221Rhttp://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpgstraight lines are not straight anymorepincushion dist.barrel dist.Radial distortion exampleMeydenbauer cameravertical lens shiftto allow direct ortho-photographsAction of projective camera on points and linesforward projection of line μbaμPBPAμB)P(AμX back-projection of linelPTPXlXTT PX x0;xlTPXx projection of pointAction of projective camera on conics and quadricsback-projection to coneCPPQTco0CPXPXCxxTTT PXx projection of quadricTPPQC**0lPPQlQT*T*T lPTiixX ? PResectioningDirect Linear Transform (DLT)iiPXx  iiPXxrank-2 matrixDirect Linear Transform (DLT)Minimal solutionOver-determined solution 5½ correspondences needed (say 6) P has 11 dof, 2 independent eq./pointsn  6 pointsApminimize subject to constraint 1p use SVDSingular Value DecompositionXXVTXVΣTXVUΣTHomogeneous least-squaresTVUΣA 1X AXminsubject tonVX solutionTVΣUXDegenerate configurations(i) Points lie on