CSCE 641:Computer Graphics Image Warping/RegistrationOutlineImage WarpingSlide 4Slide 5Parametric (global) WarpingTransformation FunctionDefinition: Image WarpingSlide 9A Transformation FunctionWarping TypesSimilarity TransformAffine TransformProjective Transform (a.k.a “perspective”)Slide 15Forward WarpingSlide 17Slide 18Slide 19Slide 20Inverse WarpingSlide 22Slide 23ResamplingResampling FilterReview: Signal Reconstruction in Freq. DomainReview: Signal Reconstruction in Spatial DomainSlide 28Point SamplingBilinear FilterSlide 31Slide 32Slide 33Anisotropic FilterComparison of Resampling FiltersInverse Warping and ResamplingSlide 37Image RegistrationWhy Image Registration?Slide 40Slide 41Slide 42Feature-based AlignmentSlide 44Slide 45Feature DetectionFeature MatchingSlide 48Slide 49Slide 50Slide 51Slide 52Feature-space Outlier RejectionSlide 54Slide 55Slide 56Slide 57Robust Estimation: A Toy ExampleSlide 59RANSAC: RANdom SAmple ConsensusRANSACRANSAC Line Fitting ExampleSlide 63Slide 64Slide 65Slide 66Slide 67Slide 68How Many Samples?Slide 70Slide 71RANSAC for Estimating Projective TransformationSlide 73Feature-based RegistrationSlide 75Slide 76Next LectureCSCE 641:Computer Graphics Image Warping/RegistrationJinxiang ChaiOutlineImage warpingImage RegistrationImage WarpingImage filtering: change range of image g(x) = h(f(x))fxhfxfxhfxImage warping: change domain of imageg(x) = f(h(x))Image WarpingImage filtering: change range of image g(x) = h(f(x))Image warping: change domain of imageg(x) = f(h(x))hf ghfgImage WarpingWhy? - texture mapping - image processing (rotation, zoom in/out, etc) - image morphing/blending - image editing - image based-modeling & renderingParametric (global) WarpingExamples of image warps:translationrotationaspectaffine perspectivecylindricalTransformation Functionf, gTransform the geometry of an image to a desired geometryDefinition: Image WarpingSource Image: Image to be used as the reference. The geometry of this image is no changedTarget Image: this image is obtained by transforming the reference image. (x,y): coordinates of points in the reference image(u,v): coordinates of points in the target imagef,g or F,G: x and y components of a transformation functionDefinition: Image WarpingControl points: Unique points in the reference and target images. The coordinates of corresponding control points in images are used to determine a transformation function.Source Image Target ImageA Transformation FunctionUsed to computer the corresponding points Source Image S(x,y) Target Image T(u,v)u = f(x,y)v = g(x.y)x = F(u,v)y = G(x.v)Warping TypesSimple mappings: - Similarity - Affine mapping - Projective mappingThese can be applied globally over a subdivision of the plane: - Piecewise affine over triangulation - Piecewise projective over quadrilaterization - Piecewise bilinear over a rectangular gridOr other, arbitrary functions can be used, e.g. - Bieer-neely warp (popular for morphs) - Store u(x,y) and v(x,y) in large arraysSimilarity Transform•A combination of 2-D scale, rotation, and translation transformations.•Allows a square to be transformed into any rotated rectangle.•Angle between lines is preserved•5 degrees of freedom (sx,sy,θ,tx,ty)•Inverse is of same form (is also similarity). Given by inverse of 3X3 matrix aboveHave the form:In matrix notation:Affine Transform•A combination of 2-D scale, rotation, shear, and translation transformations.•Allows a square to be distorted into any parallelogram.•6 degrees of freedom (a,b,c,d,e,f)•Inverse is of same form (is also affine). Given by inverse of 3X3 matrix above•Good when controlling a warp with triangles, since 3 points in 2D determined the 6 degrees of freedomHave the form:In matrix notation:Projective Transform (a.k.a “perspective”)•Linear numerator & denominator•If g=h=0, then you get affine as a special case•Allow a square to be distorted into any quadrilateral•8 degrees of freedom (a-h). We can choose i=1, arbitrarily•Inverse is of same form (is also projective). •Good when controlling a warp with quadrilaterals, since 4 points in 2D determine the 8 degrees of freedomHave the form:In matrix notation:Image WarpingxyuvGiven a coordinate transform function f,g or F,G and source image S(x,y), how do we compute a transformed image T(u,v)?S(x,y)T(u,v)Forward WarpingxyuvS(x,y)T(u,v) Forward warping algorithm: for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v)Forward WarpingxyuvS(x,y)T(u,v) Forward warping algorithm: for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v) - Any problems for forward warping?Forward WarpingxyuvS(x,y)T(u,v)Q: What if the transformed pixel located between pixels?A: Distribute color among neighboring pixels - known as “splatting”Forward Warping•Iterate over source, sending pixels to destination•Some source pixel map to multiple dest. pixels•Some dest. pixels may have no corresponding source•Holes in reconstruction •Must splat etc.for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v) xyuvForward Warping•Iterate over source, sending pixels to destination•Some source pixel map to multiple dest. pixels•Some dest. pixels may have no corresponding source•Holes in reconstruction •Must splat etc.for y = ymin to ymax for x = xmin to xmax u = f(x,y); v = g(x,y) copy pixel at source S(x,y) to T(u,v) xyuv- How to remove the holes?Inverse WarpingxyuvS(x,y)T(u,v) Inverse warping algorithm: for v = vmin to vmax for u = umin to umax x = F(u,v); y = G(u,v) copy pixel at source S(x,y) to T(u,v)Inverse WarpingxyuvS(x,y) T(u,v)Q: What if pixel comes from “between” two pixels?A: Interpolate color values from neighboring pixelsInverse Warping•Iterate over dest., finding pixels from source•Non-integer evaluation source image, resample•May oversample source•But no holes•Simpler, better than forward mappingfor v = vmin to vmax for u = umin to umax x = F(u,v); y = G(u,v) copy pixel at source S(x,y) to T(u,v) xyuvResamplingxyuvThis is a 2D signal reconstruction problem!Resampling FilterReview: Signal Reconstruction in Freq. DomainT 2T…-2T -T…0xfs(x)f(x)xFs(u)u-fmaxfmaxF(u)u-fmaxfmax)()( uboxuFsInverse Fourier transformFourier transformReview: Signal Reconstruction in Spatial Domain)sin()( xxfsT 2T…-2T