1UNC Chapel Hill - Comp 875Gabe HartManifold Kernel Regression2OutlineRegressionParametric RegressionNon-Parametric RegressionKernel RegressionBrain Volume and AgeMathematical StructureManifold Kernel RegressionWhat is a Manifold?Brain Shape and AgeMathematical StructureFacial AttractivenessConclusion3Goal: ??Given: ??Regression4{ti, yi}i=1NGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: ObservationsRegression5{ti, yi}i=1NGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observationsti, yiRegression6Regression{ti, yi}i=1NGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observationsti, yiti, yi7{ti, yi}i=1NGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observationsti, yiti, yiRegressionCan anyone name some types of regression?8RegressionTypes of Regression - ParametricLinearp t= a0a1tp ti= yi9RegressionTypes of Regression - ParametricQuadraticp t=a0a1ta2t2p ti= yi10RegressionTypes of Regression - ParametricCubicp t= a0a1ta2t2a3t3p ti= yi11RegressionTypes of Regression - ParametricPolynomial (n = 10)p t= a0a1ta2t2...antnp ti= yi12RegressionTypes of Regression – Non-ParametricGeneralizes concept of the meanex: Kernel Regression13RegressionTypes of Regression – Non-ParametricGeneralizes concept of the meanex: Kernel RegressionImage credit: B. DavisBrain Volume and Age14RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1NKernel Regression15mt ≡EY∣T =t Kernel RegressionRecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1N16RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1NKernel Regressionmt ≡EY∣T=t  =∫yP Y = y ,T =t PT =t dy17RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1Nmt ≡EY∣T=t  =∫yP Y = y ,T =t PT =t dyKernel Regression18RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1NPT =t ≡1N∑i=1NKht−tiKernel RegressionKσ = Kernel Function with bandwidth σ PY = y ,T =t ≡1N∑i=1NKht−ti Kg y− yi19Kernel ExamplesRecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1NPY = y ,T =t ≡1N∑i=1NKht−ti Kg y− yiPT =t ≡1N∑i=1NKht−tiKσ = Kernel Function with bandwidth σ Kernel Regression20RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1Nmh , gt =∫y1N∑i=1NKht−ti Kg y− yi1N∑i=1NKht−tidyKernel Regression21RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1Nmht =∑i=1NKht−ti yi∑i=1NKht−tiKernel RegressionNadaraya-Watson Kernel Estimator22RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1Nmht =∑i=1NKht−ti yi∑i=1NKht−tiKernel Regression“ Weighted average of the observations yi. Weights defined by kernel Kh which represents distribution for ti ”Nadaraya-Watson Kernel Estimator23RecallGoal: Estimate the relationship between independent random variable T and dependent random variable Y.Given: Observations{ti, yi}i= 1N“ Weighted average of the observations yi. Weights defined by kernel Kh which represents distribution for ti ”Kernel RegressionImage credit: http://en.wikipedia.org/wiki/File:Cps71_lc_mean.png24What is a Manifold?Non-planar surfaceCan be locally approximated as a planeImage credit: http://en.wikipedia.org/wiki/File:Triangles_(spherical_geometry).jpgManifold Kernel Regression25Goal: Regress relation between independent random variable T and dependant random variable P with values on manifold MGiven: Observations Brain Shape and AgeShape change is not well suited to a flat Euclidean space representationBest described by looking at transformation over timeThere is wide variability in human brain shape.Image credit: B. DavisManifold Kernel Regression{ti, pi}i =1N26Recallmht =∑i=1NKht−ti yi∑i=1NKht−tiManifold Kernel Regression“Weighted average of the observations yi.”27Recall“Weighted average of the observations yi.”Problem: Operations no longer well defined on yi so an explicate average is no longer possible.Manifold Kernel Regressionmht =∑i=1NKht−ti yi∑i=1NKht−ti28Recall“Weighted average of the observations yi.”Insight: What is an average anyway?Manifold Kernel RegressionProblem: Operations no longer well defined on yi so an explicate average is no longer possible.mht =∑i=1NKht−ti yi∑i=1NKht−ti29Recallmht =∑i=1NKht−ti yi∑i=1NKht−ti“ Weighted average of the observations yi.”Manifold Kernel RegressionInsight: What is an average anyway?30Recallmht =∑i=1NKht−ti yi∑i=1NKht−ti“ Weighted average of the observations yi.”Manifold Kernel RegressionInsight: What is an average anyway?Answer: The point in space for which the sum of distances to the observations is minimized31Recallmht =∑i=1NKht−ti yi∑i=1NKht−ti“ Weighted average of the observations yi.”Can be explicitly computed for Euclidean vector spaces.Manifold Kernel RegressionInsight: What is an average anyway?Answer: The point in space for which the sum of distances to the observations is minimizedRegression32Recallmht =∑i=1NKht−ti yi∑i=1NKht−ti“ Weighted average of the observations