1Announcements Midterm Next TUESDAY, 10/25, 5-8pm Prep page is on the web (practice exams, etc) Topical review sessions: see prep page Overall review: in class Thursday If you have a conflict, we should already know about it! Written 3 Due this Friday but fixes not due until NEXT Friday P1, P2, W1 in glookupCS 188: Artificial IntelligenceFall 2011Lecture 16: Bayes Nets IV10/18/2011Dan Klein – UC Berkeley2Approximate Inference3Approximate Inference Simulation has a name: sampling Sampling is a hot topic in machine learning,and it’s really simple Basic idea: Draw N samples from a sampling distribution S Compute an approximate posterior probability Show this converges to the true probability P Why sample? Learning: get samples from a distribution you don’t know Inference: getting a sample is faster than computing the right answer (e.g. with variable elimination)4SAF3Prior SamplingCloudySprinklerRainWetGrassCloudySprinklerRainWetGrass5+c 0.5-c 0.5+c+s 0.1-s 0.9-c+s 0.5-s 0.5+c+r 0.8-r 0.2-c+r 0.2-r 0.8+s+r+w 0.99-w 0.01-r+w 0.90-w 0.10-s+r+w 0.90-w 0.10-r+w 0.01-w 0.99Samples:+c, -s, +r, +w-c, +s, -r, +w…Prior Sampling This process generates samples with probability:…i.e. the BN’s joint probability Let the number of samples of an event be Then I.e., the sampling procedure is consistent64Example First: Get a bunch of samples from the BN:+c, -s, +r, +w+c, +s, +r, +w-c, +s, +r, -w+c, -s, +r, +w-c, -s, -r, +w Example: we want to know P(W) We have counts <+w:4, -w:1> Normalize to get approximate P(W) = <+w:0.8, -w:0.2> This will get closer to the true distribution with more samples Can estimate anything else, too What about P(C| +w)? P(C| +r, +w)? P(C| -r, -w)? Fast: can use fewer samples if less time (what’s the drawback?)CloudySprinklerRainWetGrassCSRW7Rejection Sampling Let’s say we want P(C) No point keeping all samples around Just tally counts of C as we go Let’s say we want P(C| +s) Same thing: tally C outcomes, but ignore (reject) samples which don’t have S=+s This is called rejection sampling It is also consistent for conditional probabilities (i.e., correct in the limit)+c, -s, +r, +w+c, +s, +r, +w-c, +s, +r, -w+c, -s, +r, +w-c, -s, -r, +wCloudySprinklerRainWetGrassCSRW85Sampling Example There are 2 cups. The first contains 1 penny and 1 quarter The second contains 2 quarters Say I pick a cup uniformly at random, then pick a coin randomly from that cup. It's a quarter (yes!). What is the probability that the other coin in that cup is also a quarter?Likelihood Weighting Problem with rejection sampling: If evidence is unlikely, you reject a lot of samples You don’t exploit your evidence as you sample Consider P(B|+a) Idea: fix evidence variables and sample the rest Problem: sample distribution not consistent! Solution: weight by probability of evidence given parentsBurglary AlarmBurglary Alarm10-b, -a-b, -a-b, -a-b, -a+b, +a-b +a-b, +a-b, +a-b, +a+b, +a6Likelihood Weighting11+c 0.5-c 0.5+c+s 0.1-s 0.9-c+s 0.5-s 0.5+c+r 0.8-r 0.2-c+r 0.2-r 0.8+s+r+w 0.99-w 0.01-r+w 0.90-w 0.10-s+r+w 0.90-w 0.10-r+w 0.01-w 0.99Samples:+c, +s, +r, +w…CloudySprinklerRainWetGrassCloudySprinklerRainWetGrassLikelihood Weighting Sampling distribution if z sampled and e fixed evidence Now, samples have weights Together, weighted sampling distribution is consistentCloudyRCSW127Likelihood Weighting Likelihood weighting is good We have taken evidence into account as we generate the sample E.g. here, W’s value will get picked based on the evidence values of S, R More of our samples will reflect the state of the world suggested by the evidence Likelihood weighting doesn’t solve all our problems Evidence influences the choice of downstream variables, but not upstream ones (C isn’t more likely to get a value matching the evidence) We would like to consider evidence when we sample every variable13CloudyRainCSRWMarkov Chain Monte Carlo* Idea: instead of sampling from scratch, create samples that are each like the last one. Procedure: resample one variable at a time, conditioned on all the rest, but keep evidence fixed. E.g., for P(B|+c): Properties: Now samples are not independent (in fact they’re nearly identical), but sample averages are still consistent estimators! What’s the point: both upstream and downstream variables condition on evidence.14+a +c+b+a +c-b-a +c-b8Decision Networks MEU: choose the action which maximizes the expected utility given the evidence Can directly operationalize this with decision networks Bayes nets with nodes for utility and actions Lets us calculate the expected utility for each action New node types: Chance nodes (just like BNs) Actions (rectangles, cannot have parents, act as observed evidence) Utility node (diamond, depends on action and chance nodes)WeatherForecastUmbrellaU15[DEMO: Ghostbusters]Decision Networks Action selection: Instantiate all evidence Set action node(s) each possible way Calculate posterior for all parents of utility node, given the evidence Calculate expected utility for each action Choose maximizing actionWeatherForecastUmbrellaU169Example: Decision NetworksWeatherUmbrellaUW P(W)sun 0.7rain 0.3A W U(A,W)leave sun 100leave rain 0take sun 20take rain 70Umbrella = leaveUmbrella = takeOptimal decision = leaveDecisions as Outcome Trees Almost exactly like expectimax / MDPs What’s changed?18U(t,s)Weather Weather{}U(t,r) U(l,s) U(l,r)10Evidence in Decision Networks Find P(W|F=bad) Select for evidence First we join P(W) and P(bad|W) Then we normalizeWeatherForecastW P(W)sun 0.7rain 0.3F P(F|rain)good 0.1bad 0.9F P(F|sun)good 0.8bad 0.2W P(W)sun 0.7rain 0.3W P(F=bad|W)sun 0.2rain 0.9W P(W,F=bad)sun 0.14rain 0.27W P(W | F=bad)sun 0.34rain 0.66UmbrellaUExample: Decision NetworksWeatherForecast=badUmbrellaUA W U(A,W)leave sun 100leave rain 0take sun 20take rain 70W P(W|F=bad)sun 0.34rain 0.66Umbrella = leaveUmbrella = takeOptimal decision = take2011Decisions as Outcome Trees21U(t,s)W | {b} W | {b}U(t,r) U(l,s) U(l,r){b}Value of Information Idea: compute value of acquiring evidence Can be done directly from decision network Example: buying oil drilling rights Two blocks A and B, exactly one has oil, worth k You can drill in one location Prior probabilities
View Full Document