Massachusetts Institute of TechnologyDepartment of Electrical Engineering and Computer ScienceDepartment of Mechanical Engineering6.050J/2.110J Information and Entropy Spring 2003Issued: March 30, 2003 Problem Set 7 Due: April 4, 2003Note: The quiz will be held Thursday, April 24, 2003, 12:00 noon - 1:00 PM, in Room 37-212. The quizwill be closed book except that you may bring one sheet of 8 1/2 x 11 inch paper with notes on both sides.Calculators will not be necessary but you may bring one if you wish. Material through the end of ProblemSet 8 may be covered on the quiz.Problem 1: Uncertain EmploymentYou have graduated from MIT with high honors, and have moved on to the field of politics, your passion.After many years of hard work, you eventually become advisor to the President of the United States onemployment policy. According to the class ification system you have devised there are three possibilities forthe employment status of a person:• Fully employed (100% of capacity)• Partially employed (50% of c apacity)• Unemployed (0% capacity)The President is trying to set economic policy so that the employment rate of the country is kept to 95%,as measured with your system (i.e., each partially-employed person counts as half a fully-employed person)There is some confusion, however, about whether this average is to be the goal each m onth (averaging overall the citizens) or the goal for each citizen (averaging over all months in the President’s term in office).Initially, the President says the average is to be met each month. You do not know the exact statisticsyet of the economic data, and so you express what knowledge you do have in terms of probabilities – theprobabilities p(E) (fully-employed), p(F ) (partially-employed, or as an engineer would say, fractionally-employed), and p(U) (unemployed) of the employment status of any particular citizen. Since each citizenfalls into exactly one of the categories, you know that the events E, F , and U form a partition, sop(E) + p(F ) + p(U) = 1 (7–1)Furthermore, you know that when the policy goal is achieved, the average employment will b e 0.95:1.0p(E) + 0.5p(F ) + 0.0p(U) = 0.95 (7–2)Noticing that there are many possible probability distributions consistent with these two constraints, youdecide to find the distribution that uses only the information you have. In other words, you use the principleof m aximum entropy. (This approach is consistent with your experience that usually politicians reveal theminimum amount of information about the the success of their p olicies.)Note: To do this problem, you may want to use MATLAB. If you do and then hand in your assignmenton paper, please write the MATLAB statement you used. Remember to show all work. Note that log2(x) isnot implemented correctly on all MATLAB versions, so use log(x)/log(2) instead. The commands solveand diff can be used to find where a function’s derivative is z ero; to see the help files for these operations,type help sym/solve or help sym/diff.a. Recall that all probabilities must lie in the interval between 0 and 1.1Problem Set 7 2i. What range of p(E), the probability that any one citizen is fully-employed, is consistentwith these constraints?ii. Plot the entropy of the probability distribution over this range as a function of p(E). (Ifyou turn in the problem set on paper, be sure to include this graph!)iii. Find the maximum entropy, and give the value of all three probabilities at this maximum.b. The President feels that most of the citizens would rather be fully-employed than otherwise,so he asks for another probability distribution that is consistent with the constraints but thatmaximizes p(E). Before calculating the entropy in this case, say whether it could be less than,equal to, or greater than the entropy calculated in part (a-iii) above.c. Now find the other two probabilities, and the entropy.d. Another economic advisor points out that the constraint on the average could be fulfilled byaveraging over the entire term, so there is no need to adhere to it the very first month. Hesuggests a wider range of options in assigning the probabilities. What is the minimum employment(averaged over all citizens) that the country can have in the first month? Describe in words howthis situation is achieved. What are the three probabilities in this case? What is the entropy?In this section and the next you may assume that the government has fine enough control overthe economy to adjust the number of fully employed, partially employed, or unemployed personsto whatever number it so desires.e. What is the maximum employment (averaged over all citizens) the country can have in any givenmonth? Describe in words how this situation is achieved. What are the three probabilities inthis case? What is the entropy?Problem 2: Candy RaffleYou have entered a raffle at the Cambridgeside Galleria wherein you must guess certain properties of a jarfilled with Tootsie Rolls, M&M’s, and Hershey’s Kisses. Normally you would guess only how many pieces ofcandy were in the jar, but the Galleria is wise to the ways of MIT students, and so has decided to make theraffle slightly more difficult than normal by asking for properties other than the number of pieces of candy.You know that M&M’s weigh 0.9 grams, Tootsie Rolls weigh 2.1 grams, and a Hershey’s Kiss weighs 1.5grams. You also know that Tootsie Rolls are 85% sugar and 15% other ingredients, M&M’s are 45% sugarand 55% other ingredients, and Hershey’s Kisses are 25% sugar and 75% other ingredients. The candy inthe jar has a total mass of 2300 grams, but you don’t know how many of each type are in the jar. To winthe raffle, you are asked to provide the answers to the following questions:a. What ranges of numbers for each type of candy in the jar are compatible with this total mass?b. You are told there are 1100 pieces of candy in the jar. Make no additional assumptions. Determinethose numbers of each type of candy that maximize your remaining uncertainty. Don’t worry ifyou get fractional pieces of candy. (The MATLAB solve command will not work on this problem.)c. Given the numbers you calculated, what do you expect the percentage by mass of sugar insidethe jar to be?d. Can you have 600 Tootsie Rolls? Is this consistent with the answer in part a? Explain.Problem Set 7 3Turning in Your SolutionsIf you used MATLAB for this problem set, you may have some M-files and a diary. Name the M-files withnames like ps7p1.m, ps7p2.m, and