OutlineVector SpacesHilbert SpacesMatricesLinear OperatorsOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsFunctional Analysis ReviewLorenzo Rosasco–slides courtesy of Andre Wibisono9.520: Statistical Learning Theory and ApplicationsFebruary 13, 2012L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear Operators1Vector Spaces2Hilbert Spaces3Matrices4Linear OperatorsL. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsVector SpaceA vector space is a set V with binary operations+: V × V → V and · : R × V → Vsuch that for all a, b ∈ R and v, w, x ∈ V:1v + w = w + v2(v + w) + x = v + (w + x)3There exists 0 ∈ V such that v + 0 = v for all v ∈ V4For every v ∈ V there exists −v ∈ V such that v + (−v) = 05a(bv) = (ab)v61v = v7(a + b)v = av + bv8a(v + w) = av + awExample: Rn, space of polynomials, space of functions.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsVector SpaceA vector space is a set V with binary operations+: V × V → V and · : R × V → Vsuch that for all a, b ∈ R and v, w, x ∈ V:1v + w = w + v2(v + w) + x = v + (w + x)3There exists 0 ∈ V such that v + 0 = v for all v ∈ V4For every v ∈ V there exists −v ∈ V such that v + (−v) = 05a(bv) = (ab)v61v = v7(a + b)v = av + bv8a(v + w) = av + awExample: Rn, space of polynomials, space of functions.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsInner ProductAn inner product is a function h·, ·i: V × V → R suchthat for all a, b ∈ R and v, w, x ∈ V:1hv, wi = hw, vi2hav + bw, xi = ahv, xi + bhw, xi3hv, vi > 0 and hv, vi = 0 if and only if v = 0.v, w ∈ V are orthogonal if hv, wi = 0.Given W ⊆ V, we have V = W ⊕ W⊥, whereW⊥= { v ∈ V | hv, wi = 0 for all w ∈ W }.Cauchy-Schwarz inequality: hv, wi 6 hv, vi1/2hw, wi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsInner ProductAn inner product is a function h·, ·i: V × V → R suchthat for all a, b ∈ R and v, w, x ∈ V:1hv, wi = hw, vi2hav + bw, xi = ahv, xi + bhw, xi3hv, vi > 0 and hv, vi = 0 if and only if v = 0.v, w ∈ V are orthogonal if hv, wi = 0.Given W ⊆ V, we have V = W ⊕ W⊥, whereW⊥= { v ∈ V | hv, wi = 0 for all w ∈ W }.Cauchy-Schwarz inequality: hv, wi 6 hv, vi1/2hw, wi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsInner ProductAn inner product is a function h·, ·i: V × V → R suchthat for all a, b ∈ R and v, w, x ∈ V:1hv, wi = hw, vi2hav + bw, xi = ahv, xi + bhw, xi3hv, vi > 0 and hv, vi = 0 if and only if v = 0.v, w ∈ V are orthogonal if hv, wi = 0.Given W ⊆ V, we have V = W ⊕ W⊥, whereW⊥= { v ∈ V | hv, wi = 0 for all w ∈ W }.Cauchy-Schwarz inequality: hv, wi 6 hv, vi1/2hw, wi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsInner ProductAn inner product is a function h·, ·i: V × V → R suchthat for all a, b ∈ R and v, w, x ∈ V:1hv, wi = hw, vi2hav + bw, xi = ahv, xi + bhw, xi3hv, vi > 0 and hv, vi = 0 if and only if v = 0.v, w ∈ V are orthogonal if hv, wi = 0.Given W ⊆ V, we have V = W ⊕ W⊥, whereW⊥= { v ∈ V | hv, wi = 0 for all w ∈ W }.Cauchy-Schwarz inequality: hv, wi 6 hv, vi1/2hw, wi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsInner ProductAn inner product is a function h·, ·i: V × V → R suchthat for all a, b ∈ R and v, w, x ∈ V:1hv, wi = hw, vi2hav + bw, xi = ahv, xi + bhw, xi3hv, vi > 0 and hv, vi = 0 if and only if v = 0.v, w ∈ V are orthogonal if hv, wi = 0.Given W ⊆ V, we have V = W ⊕ W⊥, whereW⊥= { v ∈ V | hv, wi = 0 for all w ∈ W }.Cauchy-Schwarz inequality: hv, wi 6 hv, vi1/2hw, wi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsNormA norm is a function k · k : V → R such that for all a ∈ Rand v, w ∈ V:1kvk > 0, and kvk = 0 if and only if v = 02kavk = |a| kvk3kv + wk 6 kvk + kwkCan define norm from inner product: kvk = hv, vi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsNormA norm is a function k · k : V → R such that for all a ∈ Rand v, w ∈ V:1kvk > 0, and kvk = 0 if and only if v = 02kavk = |a| kvk3kv + wk 6 kvk + kwkCan define norm from inner product: kvk = hv, vi1/2.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsMetricA metric is a function d: V × V → R such that for allv, w, x ∈ V:1d(v, w) > 0, and d(v, w) = 0 if and only if v = w2d(v, w) = d(w, v)3d(v, w) 6 d(v, x) + d(x, w)Can define metric from norm: d(v, w) = kv − wk.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsMetricA metric is a function d: V × V → R such that for allv, w, x ∈ V:1d(v, w) > 0, and d(v, w) = 0 if and only if v = w2d(v, w) = d(w, v)3d(v, w) 6 d(v, x) + d(x, w)Can define metric from norm: d(v, w) = kv − wk.L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsBasisB = {v1, . . . , vn} is a basis of V if every v ∈ V can beuniquely decomposed asv = a1v1+ · · · + anvnfor some a1, . . . , an∈ R.An orthonormal basis is a basis that is orthogonal(hvi, vji = 0 for i 6= j) and normalized (kvik = 1).L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsBasisB = {v1, . . . , vn} is a basis of V if every v ∈ V can beuniquely decomposed asv = a1v1+ · · · + anvnfor some a1, . . . , an∈ R.An orthonormal basis is a basis that is orthogonal(hvi, vji = 0 for i 6= j) and normalized (kvik = 1).L. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear Operators1Vector Spaces2Hilbert Spaces3Matrices4Linear OperatorsL. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsHilbert Space, overviewGoal: to understand Hilbert spaces (complete innerproduct spaces) and to make sense of the expressionf =∞Xi=1hf, φiiφi, f ∈ HNeed to talk about:1Cauchy sequence2Completeness3Density4SeparabilityL. Rosasco Functional Analysis ReviewOutlineVector SpacesHilbert SpacesMatricesLinear OperatorsCauchy SequenceRecall: limn→∞xn= x if …
View Full Document
Unlocking...