Massachusetts Institute of TechnologyDepartment of Electrical Engineering & Computer Science6.041/6.431: Probabilistic Systems Analysis(Fall 2008)Problem Set 11Due: December 3rd, 2008Topics: Bayesian Least Mean Squares , Bayesian Linear Least Mean SquaresEstimation, Classical Parameter Estimation1. Suppose that the joint distribution of random variables X and Y is given by:fX,Y(x, y) =1/4 if (x, y) ∈ A0 if (x, y) /∈ Awhere A is the shaded area in Figure 1.(a) You have the choice of estimating Y based on observations of X or to estimate X based onobservations of Y. Which option should you pick to minimize the conditional mean squarederror of your estimation for the worst choice of your measured variable?(b) Compute and plot the Bayesian Least Mean Square estimate of X based on observation ofY.(c) Compute and plot the Bayesian Least Mean Square estimate of Y based on observation ofX.(d) Compute the conditional mean squared errors of your answers in parts (b) and (c). For whatobservation value(s) of Y does the LMS estimate of X have the worst error? Similarly, forwhat obs ervation value(s) of X does the LMS estimate of Y have the worst error?(e) Compute the mean squared errors of your answers in parts (b) an d (c) and compare theresults?(f) What problems do you expect to encounter, if any, if you repeat parts (b) and (c) using theMAP rule for estimation instead of the LMS.Figure 1: The joint distrib ution of random variables X and Y.2. Continuous random variables X and Y h ave a joint PDF given byfX,Y(x, y) =c if (x, y) belongs to the closed sh ad ed r egion0 otherwiseCompiled November 27, 2008 Page 1 of 4Massachusetts Institute of TechnologyDepartment of Electrical Engineering & Computer Science6.041/6.431: Probabilistic Systems Analysis(Fall 2008)yx211 2(a) The value of X will be revealed to us; we have to design an estimator g(X) of Y thatminimizes the conditional expectation E[(Y − g(X))2|X = x], for all x, over all possibleestimators. Provide a plot of the optimal estimator as a function of its argument.(b) Let g∗(X) be the optimal estimator of part (a). Find the numerical value of E[g∗(X)] andvar(g∗(X))?(c) Find the least mean squared estimation error E[(Y −g∗(X))2]. Is that the same as E[var(Y |X)]?(d) Find var(Y ).(e) Let l∗(X) be the optimal linear LMS estimator. Plot l∗(X) and find th e numerical value ofE[l∗(X)] and var(l∗(X))?(f) The mean squared error of the linear LMS estimator is defined as E[(Y − l∗(X))2]. Whichdo you think will be larger, E[(Y −g∗(X))2] or E[(Y −l∗(X))2]. Calculate E[(Y −l∗(X))2]and verify your answer.3. (Depends on Lecture Materia l presented on 12/1) According to Planck’s Law, a bodyat temperature Θ radiates photons at a given wavelength. This problem will have you estimateΘ, which is fixed but unknown. The PMF for th e number of photons K in a given wavelengthrange and a fixed time interval of one second is given by,pK(k; θ) =1Z(θ)e−kθ, k = 0, 1, 2, ...Z(θ) is a normalization factor for the probability distribution (Th e p hysicists call it the partitionfunction). You are given the task of determining the temperature of the body to two significantdigits by photon counting in non-overlapping time intervals of duration one second. The photonemissions in non -overlapping time intervals are s tatistically ind ependent from each other.(a) Determine the normalization factor Z(θ).(b) Compute the expected value of th e photon numb er measured in any 1 second time interval,µK= Eθ[K] and its variance, varθ(K) = σ2K.(c) You count the number kiof photons detected in n non-overlapping 1 second time intervals.Find the maximum likelihood estimator,ˆΘn, for temperature Θ . Note, it might be u s efulto introduce the average photon number sn=1nPni=1ki. And in order to keep the analysissimple we assume that the bo dy is hot, i.e. Θ ≫ 1.In the following questions we wish to estimate the mean of the photon count in a one secondtime interval using the estimatorˆK, which is given by,Compiled November 27, 2008 Page 2 of 4Massachusetts Institute of TechnologyDepartment of Electrical Engineering & Computer Science6.041/6.431: Probabilistic Systems Analysis(Fall 2008)ˆK =1nnXi=1Ki.(d) Find the number of samples n for which the noise to signal ratio forˆK, (i.e.,σˆKµˆK), is 0.01.(e) Find the 95% confidence interval for the mean photon count estimate for the situation inpart (d ). (You may use the central limit theorem.)Swallowed Buffon’s Needle: A doctor is treating a patient who has accidentally swallowed aneedle. The key factor in whether to operate on the patient is the true length of the needle, whichis not known exactly but is assumed to be uniformly distributed between 0 and ℓ > 0. Whilethe needle may show up on an X-ray, the doctor recognizes that the random orientation of theneedle within the patient’s stomach implies the needle’s length on film could be misleading. Thedoctor has asked you to analyze this scenario and form an estimate of the needle’s true lengthbased on its projected length in the X-ray.We attach a 3-dimensional coordinate system to the problem such that the origin is at themidpoint of the needle, an d the needle lies parallel to the vertical-axis. Our view, then, isdescribed by the random pair of angles (Θ, Ψ), where Θ ∈ [0, 2π) is the azimuth angle (itdescribes the orientation in the horizontal-plane) and Ψ ∈ [−π/2, π/2] is the elevation angle (itdescribes the orientation out of the horizontal-plane).4.ΘΨl/2−l /2needleviewLet X be the tru e length of the needle, Y be the projected length of the needle, Θ be the azimuthangle, and Ψ be the elevation angle. We see that the azimuth angle does not affect the length ofthe p rojection and that the proj ected length is given byY = X cos Ψ.Compiled November 27, 2008 Page 3 of 4Massachusetts Institute of TechnologyDepartment of Electrical Engineering & Computer Science6.041/6.431: Probabilistic Systems Analysis(Fall 2008)It is evident that the problem is symmetric about the xy-plane. Hence an elevation angle ofΨ is equivalent to an elevation angle of −Ψ. Thus, we can define W to be W = |Ψ|, so W isuniformly-distributed between 0 and π/2.(a) Determine the least-squares estimate of the needle’s true length given the value of the nee-dle’s projected length in a single X-ray. In other words, if X were to denote the needle’s truelength and Y its length as measured from th e resulting X-ray,
View Full Document
Unlocking...