EE226: Random Processes in Systems Fall’06Problem Set 1 — Sept, 14Lecturer: Jean C. Walrand GSI: Assane GueyeThis problem set essentially reviews notions of conditional expectation, conditional dis-tribution, and Jointly Gaussian random variables. Not all exercises are to be turned in. Onlythose with the sign F are due on Thursday, September 21 at the beginning of the class.Although the remaining exercises are not graded, you are encouraged to go through them.We will discuss some of the exercises during discussion sections.Please feel free to point out errors and notions that need to be clarified.We give solutions for only the problems that were required to be turned in.You are encouraged to try the other problems and discuss your solutions duringoffice hours if needed.Exercise 1: Linear AlgebraIn this exercise, we will review linear system of equations by studying the conditions underwhich there exists a solution. Some exercises require using matlab or other math software.The following matlab functions can be useful in this exercise: inv, det, pinv, eig, svd, range,rank, rref, null and the ones you might find yourself...Remember: to get help on one com-mand, type help command.Let Ax = b be a system of m linear equations and n unknowns, where A is the m×n-matrixof coefficients, b the vector of known terms, x is the vector of unknowns.If the rank(A)=r, then there exist at least one r × r sub matrix of A that is not singular,we choose one and call it M. We call the equations that correspond to the rows of M mainequations and side equations the others. The unknowns that correspond to columns of Mare called main unknowns and the others are called sides unknowns.The characteristic matrix, associated with a particular side equation, is the matrix formedby adding to the main matrix:1. a row at the bottom, with the coefficients of the main unknowns in that side equation2. a column at the right, with the known terms of the main equations and of the knownterm of that side equation.The characteristic determinant is the determinant of the characteristic matrix. So there areas much characteristic determinants as side equations.In the following examples, in addition to specific questions (for each example), you areasked to:1. find the main and side equations, the main and side unknowns1-1EE226 Problem Set 1 — Sept, 14 Fall’062. determine whether the matrix A is full rank or not3. give a guess on how many solutions exist4. compute all the characteristic determinants5. compute the row echelon form of (A|b)6. confirm or infirm your guess in 3)7. write down the solution(s) if there is any.Example 1:Cramer systemA =2 1 −12 0 11 −1 0b =230Give answers to the questions 1-7.What can you conclude for such kind of linear system?Example 2: Fat matrixA =µ1 −3 11 1 2¶b =µ13¶Give answers to the questions 1-7.Now suppose that b is in the formb =µb1b2¶For what value(s) of b1and b2a solution exits?What can you conclude for such kind of linear system?Example 3: Tall matrixA =0 2 12 −2 11 0 11 1 −1b1=200510b2=6242Give answers to the questions 1-7 for both b1and b2.What can you conclude for such kind of linear system?We will continue this exercise in example 5.1-2EE226 Problem Set 1 — Sept, 14 Fall’06Example 4:Rank deficient matrixA =1 0 11 −1 24 −1 5b1=4517b1=4510Give answers to the questions 1-7 for both b1and b2.What can you conclude for such kind of linear system?Example 5:Least-Square SolutionRe-consider example 3.For which bii = 1 , 2 there was a solution?Can you write bii = 1 , 2 as a linear combination of the columns of A?.Conclude!Consider the bifor which there is no solution call it bo. We will compute an approximativesolution. To be more explicit, we will compute the Least-Square solution. In fact sinceAx −bo6= 0, k Ax − bok> 0. We would like to find y ∈ Rnsuch thatk Ay − bok≤k Au − bok ∀u ∈ Rn.Method1:Compute the projection of bointo the space of the rows of A, call it proj(bo). Now answerthe 7-questions for the system Ax = proj(bo).Is there a solution? If yes write down the solution(s).Method2:Answer the questions 1), 2), and 3) for the system A∗Ax = A∗bo.The matrix A∗A on the left-hand side is a square matrix, which is invertible if A has fullcolumn rank (that is, if the rank of A is n). In that case, the solution of the system of linearequations is unique and given byx = (A∗A)−1A∗boCompute the solution if it exists.The matrix (A∗A)−1A∗is called the pseudo inverse of A. It is returned by the matlab func-tion pinv applied to A. Use this function to verify the previous result.Method3:Compute a svd of A = UΣV∗. Considering Σ+(the transpose of Σ with every nonzero entryreplaced by its reciprocal) as the inverse of Σ, compute the ”solution” of the linear systemUΣV∗x = bo.Did the results of all methods agree?Which method is less/more computationally expensive?1-3EE226 Problem Set 1 — Sept, 14 Fall’06Probability Space and Random VariableMeasurable SpaceA measurable space (Ω, B) is a pair consisting of a sample space Ω together with a σ-fieldB of subsets of Ω (also called the event space). A σ-field or σ-algebra B is a collection ofsubsets of Ω with the following properties:Ω ∈ B (1.1)If F ∈ B, then Fc= {ω : ω /∈ F } ∈ B (1.2)If Fi∈ B, i = 1, 2, . . . , then ∪iFi∈ B (1.3)Exercise 1.1. Use de Morgan’s law, show thatIf Fi∈ B, i = 1, 2, . . . , then ∩iFi∈ BAn event space is a collection of subsets of a sample space (called events by virtue ofbelonging to the event space) such that any countable sequence of set theoretic op erations(union, intersection, complement) on events produces other events.Exercise 1.2. What is the largest possible σ-field of Ω? What is the smallest possibleσ-field of Ω?Probability SpacesA probability space (Ω, B, P ) is a triple consisting of a sample space Ω, a σ-field B of subsetsof Ω, and a probability measure P (F ) defined on the σ-field; that is, P (F ) assigns a realnumber to every member F of B so that the following conditions are satisfied:NonnegativityP (F ) ≥ 0 ∀F ∈ B, (1.4)NormalizationP (Ω) = 1, (1.5)Countable AdditivityIf Fi∈ B, i = 1, 2, . . . are disjoints, thenP (∪∞i=1Fi) =∞Xi=1P (Fi) (1.6)A set function P satisfying only 1.4 and 1.6 but not necessarily 1.5 is called a measure, andthe