EE363 Prof. S. BoydEE363 homework 41. Estimating an unknown constant from repeated measureme nts. We wish to estimatex ∼ N (0, 1) from measurements yi= x + vi, i = 1, . . . , N, where viare IID N (0, σ2),uncorrelated with x. Find an explicit expression for the MMSE estimator ˆx, and theMMSE error.2. Estimator e rror variance and correlation coefficient. Suppo se (x, y) ∈ R2is Gaussian,and let ˆx denote the MMSE estimate of x given y, and ¯x denote the expected value ofx. We define the relative mean square estimator error as η = E(ˆx − x)2/ E(¯x − x)2.Show that η can be expressed as a function of ρ, the correlation coefficient of x and y.Does your answer make sense?3. MMSE predictor and interpolator. A scalar time series y(0), y(1), . . . is modeled asy(t) = a0w(t) + a1w(t − 1) + · · · + aNw(t − N),where w(−N), w(−N + 1), . . . ar e IID N (0, 1). The coefficients a0, . . . , aNare known.(a) Predicting nex t value from current value. Find the MMSE predictor o f y(t + 1)based on y(t). (Note: we really mean based on just y(t), and not based ony(t), y(t − 1), . . .) Your answer should be as explicit as possible.(b) MMSE interpolator. Find the MMSE predictor of y(t) (for t > 1) based (only)on y(t − 1) and y(t + 1) (for t ≥ 1). Your answer should be as explicit as possible.4. Estimating initial subpopulations from total growth observations. A sample that con-tains three types of bacteria (called A, B, and C) is cultured, and the to t al bacteriapopulation is measured every hour. The bacteria populations grow, independently ofeach other, exponentially with different growth rates: A grows 2 % per hour, B grows5% per hour, and C grows 10% per hour. The goal is to estimate the initial bacteriapopulations based on t he measurements of total population.Let xA(t) denote the population of bacteria A after t hours (say, measured in grams),for t = 0, 1, . . ., and similarly for xB(t) and xC(t), so thatxA(t + 1) = 1.02xA(t), xB(t + 1) = 1.05xB(t), xC(t + 1) = 1.10xC(t).The total population measurements are y(t) = xA(t) + xB(t) + xC(t) + v(t), where v(t)are IID, N (0, 0.25). (Thus the total population is measured with a standard deviationof 0.5).The prior information is that xA(0), xB(0), xC(0) (which are what we want to estimate)are IID N (5, 2). (Obviously the Ga ussian model is not completely accurate since it1allows the initial populations to be negative with some small probability, but we’llignore that.)How long will it be (in hours) before we can estimate xA(0) with a mean square errorless than 0.01? How long for xB(0)? How long for xC(0)?5. Sensor selection. Suppose 5 scalar measurements y1, . . . , y5are related to 3 scalarvariables x1, x2, x3byy = Ax + v, A =1 2 3−1 1 31 −2 0−1 0 −31 2 −3,where viare IID N (0, 0.01). Thus the measurement errors have standard deviation0.1. The variable x ∈ R3is deterministic but unknown.You are to find an estimator ˆx = By that satisfies the fo llowing specifications:• BA = I. This means that ˆx = x when v = 0, and also that the estimator isunbiased, i.e., E ˆx = x.• E kˆx − xk2≤ 0 .0 05.Among all matrices B that satisfy these specifications, you are to find one that mini-mizes the number of nonzero columns.To understand the practical meaning of this, note that if the jt h column o f B is zero,then the estimate ˆx does not use the jth measurement, so the jth sensor is not needed.Thus we are seeking the smallest sensor configuration (in terms of number of sensorsrequired) that can be used to estimate x with mean square error not exceeding 0.005.Make sure to describe exactly how you are going to solve the problem, as well as givingyour explicit solution (i.e., B). If no B satisfies the specifications, say so, and showwhy.6. MMSE estimation example. We wish to estimate x ∈ Rn, given a set of measurementsy ∈ Rm, wherey = Ax + v, x ∼ N (¯x, Σx), v ∼ N (0, Σv),with x and v independent. We’ll consider the specific case with n = 4, m = 6 , andmeasurement matrixA =2 3 −1 41 0 0 −22 1 1 0−3 −1 0 01 0 0 −10 −1 1 0.2The prior distribution of x is given by ¯x = (2, −1, 0.5 , −3),σ21= 2, σ22= 5, σ23= 1, σ24= 2,andρ12= −0.1, ρ13= 0.025, ρ23= −0.01.Here σiis the standard deviation of xi, and ρijis the correlation coefficient between xiand xj; correlation coefficients not given are zero. The noise statistics are characterizedby standard deviations˜σ21= 2, ˜σ22= 1, ˜σ23= 2, ˜σ24= 3, ˜σ25= 2, ˜σ26= 1,and correlation coefficients˜ρ13= −0.25, ˜ρ24= 0.5, ˜ρ35= 0.3, ˜ρ46= −0.04,with other correlation coefficient s zero.In this problem you will compare the performance of two estimat ors. The first is thesimple pseudo-inver se estimator, ˆxpinv= A†y, (as studied in EE263). The second isthe MMSE estimator, denoted ˆxmmse. You will first make some predictions about theperformance of the two estimators, based on some linear algebra calculations; then,you will carry out simulations of the two to verify your calculations.(a) Find E kˆxpinv−xk2and E kˆxmmse−xk2, explaining your method. Briefly commenton the results.(b) Let E = {ˆx − x | (ˆx − x)TΣ−1(ˆx − x) ≤ α} be the 90% confidence ellipsoid for theMMSE estimation error ˆxmmse− x. Find Σ and α.(c) Generate 1000 samples of (x,v) pairs from the distributions described above. Foreach sample, find ˆxmmse− x and ˆxpinv− x. Plot the empirical distributions (his-tograms) of kˆxmmse− xk2and kˆxpinv− xk2. (Plot the histograms on the samehorizontal axis, so they are easier to compare.) Verify that the empirical averagesof these are close to values predicted in part (a).(d) For how many o f yo ur 1000 samples from part (c) does ˆxmmse− x fall inside the90% confidence ellipsoid E found in part (b)? (O bviously this number should benear 900. If the number turns out to exactly 900, change your seed and run thesimulations again.)Some Matlab hints.• sqrtm(A) finds the (matrix) square root of matrix A.• randn(n,1) generates a column vector of dimension n drawn from a normal dis-tribution N (0, I). (Do not confuse randn with rand, which draws the entr ies ofthe vector from a uniform distribution on [0, 1].)3• hist(y,M) bins the elements of y into M equally spaced bins and plots the results.(For 1000 samples, M around 50 gives a reasonable plot.)• chi2inv(p,n) returns