How is project #1 going?Last LectureTodayOverviewClose-upLet’s take a photoSlow-motion replaySlow-motion replayImage formation processWhy is this hard?Multiple possible solutionsNatural image statisticsBlury images have different statisticsParametric distributionThree sources of informationVariational Bayesian methodVariational Bayesian methodOverview of algorithmPreprocessingInitializationInferring the kernel: multiscale methodImage ReconstructionResults on real imagesImage artifacts & estimated kernelsSummary Image WarpingImage WarpingParametric (global) warpingParametric (global) warpingScalingScalingScaling2-D Rotation2-D Rotation2x2 Matrices2x2 Matrices2x2 Matrices2x2 MatricesAll 2D Linear TransformationsHomogeneous CoordinatesHomogeneous CoordinatesHomogeneous CoordinatesTranslationHomogeneous CoordinatesBasic 2D TransformationsAffine TransformationsProjective TransformationsMatrix Composition2D image transformationsRecovering TransformationsTranslation: # correspondences?Euclidian: # correspondences?Affine: # correspondences?Projective: # correspondences?Example: warping triangleswarping triangles (Barycentric Coordinaes)Image warpingForward warpingForward warpingInverse warpingInverse warpingHow is project #1 going?Edge DetectionLast LectureFilteringPyramidTodayMotion DeblurImage TransformationRemoving Camera Shake from a Single PhotographRob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis and William T. FreemanMassachusetts Institute of Technology andUniversity of Torontohttp://people.csail.mit.edu/fergus/research/deblur.htmlOverviewOriginalOur algorithmClose-upOriginalNaïve Sharpening Our algorithmLet’s take a photoBlurry resultSlow-motion replaySlow-motion replayMotion of cameraImage formation process= ⊗Blurry imageSharp imageBlur kernelInput to algorithmDesired outputConvolutionoperatorModel is approximationWhy is this hard?Simple analogy:11 is the product of two numbers.What are they?No unique solution: 11 = 1 x 1111 = 2 x 5.511 = 3 x 3.667 etc…..Need more information !!!!Multiple possible solutions=⊗Blurry imageSharp imageBlur kernel=⊗=⊗Natural image statisticsHistogram of image gradientsCharacteristic distribution with heavy tailsBlury images have different statisticsHistogram of image gradientsParametric distributionHistogram of image gradientsUse parametric model of sharp image statisticsThree sources of information1. Reconstruction constraint:=⊗Input blurry imageEstimated sharp imageEstimatedblur kernel2. Image prior:3. Blur prior:Positive&SparseDistribution of gradientsVariational Bayesian methodBased on work of Miskin & Mackay 2000Keeps track of uncertainty in estimates of image and blur by using a distribution instead of a single estimateHelps avoid local maxima and over-fittingVariationalBayesVariational Bayesian methodMaximum a-Posteriori (MAP)Pixel intensityScoreObjective function for a single variableOverview of algorithmInput image1. Pre-processing2. Kernel estimation- Multi-scale approach3. Image reconstruction- Standard non-blind deconvolution routinePreprocessingConvert tograyscaleInput imageRemove gammacorrectionUser selects patchfrom imageBayesian inference too slow to run on whole imageInfer kernel from this patchInitializationInput imageInitialize 3x3 blur kernelInitial blur kernelBlurry patchInitial image estimateConvert tograyscaleRemove gammacorrectionUser selects patchfrom imageInferring the kernel: multiscale methodInput imageLoop over scalesVariationalBayesUpsampleestimatesUse multi-scale approach to avoid local minima:Initialize 3x3 blur kernelConvert tograyscaleRemove gammacorrectionUser selects patchfrom imageImage ReconstructionInput imageFull resolutionblur estimateNon-blind deconvolution(Richardson-Lucy)DeblurredimageLoop over scalesVariationalBayesUpsampleestimatesInitialize 3x3 blur kernelConvert tograyscaleRemove gammacorrectionUser selects patchfrom imageResults on real imagesSubmitted by people from their own photo collectionsType of camera unknown Output does contain artifacts– Increased noise– RingingCompares well to existing methodsOriginal photographBlur kernelOur outputOriginal photographMatlab’s deconvblindOriginal photographMatlab’s deconvblindPhotoshop sharpen more2 4 6 8 10 122468Our outputBlur kernel2 4 6 8 10 122468Original photograph123456789Our outputBlur kernelOriginal photographOur outputBlur kernelMatlab’s deconvblindOriginal photographOur outputBlur kernelClose-up of birdOriginalOur outputOriginal photographOur outputBlur kernelImage artifacts & estimated kernelsBlur kernelsImage patternsNote: blur kernels were inferred from large image patches,NOT the image patterns shownSummary Method for removing camera shake from real photographsFirst method that can handle complicated blur kernelsUses natural image statistics Non-blind deconvolution currently simplisticImage Warping• image filtering: change range of image• g(x) = T(f(x))fxTfxfxTfx• image warping: change domain of image• g(x) = f(T(x))Image Warping• image filtering: change range of image• g(x) = T(f(x))TT• image warping: change domain of image• g(x) = f(T(x))Parametric (global) warping• Examples of parametric warps:translationrotationaspectaffineperspectivecylindricalParametric (global) warping• Transformation T is a coordinate-changing machine:•p’ = T(p)• What does it mean that T is global?– Is the same for any point p– can be described by just a few numbers (parameters)• Let’s represent T as a matrix:•p’ = MpTp = (x,y) p’ = (x’,y’)⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxyxM''Scaling• Scaling a coordinate means multiplying each of its components by a scalar• Uniform scaling means this scalar is the same for all components:× 2• Non-uniform scaling: different scalars per component:ScalingX × 2,Y × 0.5Scaling• Scaling operation:• Or, in matrix form:byyaxx==''⎥⎦⎤⎢⎣⎡⎥⎦⎤⎢⎣⎡=⎥⎦⎤⎢⎣⎡yxbayx00''scaling matrix SWhat’s inverse of S?2-D Rotationθ(x, y)(x’, y’)x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)2-D Rotation•This is easy to capture in matrix form:•Even though sin(θ) and cos(θ) are nonlinear functions of θ,– x’ is a linear combination of x and y– y’ is a linear combination of x and y•What is the inverse transformation?– Rotation by –θ– For rotation matrices()()() ()⎥⎦⎤⎢⎣⎡⎥⎦⎤⎢⎣⎡−=⎥⎦⎤⎢⎣⎡yxyxθθθθcossinsincos''TRR=− 1R2x2 Matrices• What types of transformations