DOC PREVIEW
BU EECE 522 - Intro to Estimation

This preview shows page 1-2-3-4-5 out of 15 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 15 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

Ch. 1 Introduction to EstimationAn Example Estimation Problem: DSB RxDiscrete-Time Estimation ProblemPDF of Estimate1.2 Mathematical Estimation ProblemEx. Estimating a DC Level in Zero Mean AWGNEx. Modeling Data with Linear TrendTypical Assumptions for Noise ModelClassical vs. Bayesian Estimation Approaches1.3 Assessing Estimator PerformanceEquivalent View of Assessing PerformanceExample: DC Level in AWGNTheoretical Analysis vs. SimulationsCourse Goal = Find “Optimal” Estimators1/15Ch. 1 Introduction to Estimation2/15An Example Estimation Problem: DSB RxS( f )f oM( f )f f –f o)2cos()(),;(ooootftmftsφπφ+=AudioAmpBPF&Amp)()()( twtstx+=X)ˆˆ2cos(ootfφπ+Est. Algo.Electronics Adds Noise w(t) (usually “white”)oofφˆ&ˆf )(ˆfMOscillatorw/oofφˆ&ˆGoal: GivenFind Estimates(that are optimal in some sense))(),;()( twftstxoo+=φDescribe with Probability Model: PDF & Correlation3/15Discrete-Time Estimation ProblemThese days, almost always work with samples of the observed signal (signal plus noise):][],;[][ nwfnsnxoo+=φOur “Thought” Model: Each time you “observe” x[n] it contains same s[n] but different “realization” of noise w[n], so the estimate is different each time.oofφˆ&ˆare RVsOur Job: Given finite data set x[0], x[1], … x[N-1] Findestimator functions that map data into estimates:)(])1[,],1[],0[(ˆ)(])1[,],1[],0[(ˆ2211xxgNxxxggNxxxgfoo=−==−=……φThese are RVs… Need to describe w/ probability model4/15PDF of EstimateBecause estimates are RVs we describe them with a PDF…Will depend on: 1. structure of s[n]2. probability model of w[n]3. form of est. function g(x))ˆ(ofpofˆofMean measures centroidStd. Dev. & Variance measure spreadDesire:()small }ˆ{ˆ}ˆ{22ˆ=−==oofoofEfEffEoσ5/151.2 Mathematical Estimation ProblemGeneral Mathematical Statement of Estimation Problem:For… Measured Data x = [ x[0] x[1] … x[N-1] ]Unknown Parameter θ = [θ1θ2…θp]θ is Not Random x is an N-dimensional random data vectorQ: What captures all the statistical information needed for an estimation problem ?A: Need the N-dimensional PDF of the data, parameterized by θ);( θxpIn practice, not given PDF!!!Choose a suitable model• Captures Essence of Reality• Leads to Tractable AnswerWe’ll use p(x;θ) to find )(ˆxθ g=6/15Ex. Estimating a DC Level in Zero Mean AWGN]0[]0[ wx+=θConsider a single data point is observedGaussianzero meanvariance σ2~ N(θ, σ2)So… the needed parameterized PDF is: p(x[0];θ) which is Gaussian with mean of θSo… in this case the parameterization changes the data PDF mean:θ1p(x[0];θ1)x[0]θ2p(x[0];θ2)x[0]θ3p(x[0];θ3)x[0]7/15Ex. Modeling Data with Linear TrendSee Fig. 1.6 in TextLooking at the figure we see what looks like a linear trend perturbed by some noise…So the engineer proposes signal and noise models:[]][][],;[nwBnAnxBAns++="#"$%Signal Model: Linear Trend Noise Model: AWGN w/ zero meanAWGN = “Additive White Gaussian Noise”“White” = x[n] and x[m] are uncorrelated for n ≠m{}Iwwww2))((σ=−−TE8/15Typical Assumptions for Noise Model• W and G is always easiest to analyze– Usually assumed unless you have reason to believe otherwise– Whiteness is usually first assumption removed– Gaussian is less often removed due to the validity of Central Limit Thm• Zero Mean is a nearly universal assumption – Most practical cases have zero mean– But if not…µ+=][][ nwnwzmNon-Zero Mean of µ Zero Mean Now group into signal model• Variance of noise doesn’t always have to be known to make an estimate– BUT, must know to assess expected “goodness” of the estimate– Usually perform “goodness” analysis as a function of noise variance (or SNR = Signal-to-Noise Ratio)– Noise variance sets the SNR level of the problem9/15Classical vs. Bayesian Estimation ApproachesIf we view θ (parameter to estimate) as Non-Random→ Classical EstimationProvides no way to include a priori information about θIf we view θ (parameter to estimate) as Random → Bayesian EstimationAllows use of some a priori PDF on θThe first part of the course: Classical Methods• Minimum Variance, Maximum Likelihood, Least Squares Last part of the course: Bayesian Methods• MMSE, MAP, Wiener filter, Kalman Filter10/151.3 Assessing Estimator PerformanceCan only do this when the value of θ is known:• Theoretical Analysis, Simulations, Field Tests, etc.is a random variableRecall that the estimate)(ˆxg=θThus it has a PDF of its own… and that PDF completelydisplays the quality of the estimate.Illustrate with 1-D parameter caseθθˆ)ˆ(θpOften just capture quality through mean and variance of )(ˆxg=θDesire:()small }ˆ{ˆ}ˆ{22ˆˆ=−===θθσθθθθEEEmIf this is true: say estimate is “unbiased11/15Equivalent View of Assessing Performance)ˆ(ˆee +=−=θθθθDefine estimation error:RV RV Not RVCompletely describe estimator quality with error PDF: p(e)p(e)eDesire:(){}small }{0}{22=−===eEeEeEmeeσIf this is true: say estimate is “unbiased12/15Example: DC Level in AWGNModel:x 1,,1,0],[][−=+= NnnwAn …Gaussian, zero mean, variance σ2White (uncorrelated sample-to-sample)PDF of an individual data sample: −−=2222)][(exp21])[(σπσAixixpUncorrelated Gaussian RVs are Independent… so joint PDF is the product of the individual PDFs:−−=−−=∑∏−=−=21022/2102222)][(exp)2(12)][(exp21)(σπσσπσNnNNnAnxAnxp x( property: prod of exp’s gives sum inside exp )13/15Each data sample has the same mean (A), which is the thing we are trying to estimate… so, we can imagine trying to estimate A by finding the sample meanof the data:Statistics∑−==10][1ˆNnnxNAProb. TheoryLet’s analyze the quality of this estimator…• Is it unbiased? AAEixENnxNEAEnANn=⇒==∑∑=−=}ˆ{]}[{1][1}ˆ{10#$%Yes! Unbiased!NANNNnxNnxNANnNnNn222102210210)ˆvar(1])[var(1][1var)ˆvar(σσσ=⇒====∑∑∑−=−=−=Can make var small by increasing N!!!Due to Indep.(white & Gauss. ⇒ Indep.)• Can we get a small variance?14/15Theoretical Analysis vs. Simulations• Ideally we’d like to be always be able to theoretically analyze the problem to find the bias and variance of the estimator– Theoretical results show how performance depends on the problem specifications• But sometimes we make use of


View Full Document

BU EECE 522 - Intro to Estimation

Download Intro to Estimation
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Intro to Estimation and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Intro to Estimation 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?