1CS 188: Artificial IntelligenceSpring 2006Lecture 20: Utilities4/4/2006Dan Klein – UC BerkeleyRecap: HMMs Hidden Markov models (HMMs) Underlying Markov chain over states X You only observe outputs (effects) E at each time step Want to reason about the hidden states X given observations EXTX2E1X1X3X4E2E3E4ET2Recap: Speech Recognition Observations are acoustic measurements Real systems: 39 MFCC coefficients Real numbers, modeled with mixtures of multidimensional Gaussians Your projects: 2 real numbers (formant frequencies) Discretized values, discrete conditional probse12e13e12e14e14Speech Recognition States indicate which part of which word we’re speaking Each word broken into phonemes Real systems: context-dependent sub-phonemes Your projects: just one state per phoneme Example: Yes/No recognizeryεsn o</s><s>3Speech Recognition Emission probs: distribution over acoustic observations for each phoneme How to learn these? See project 3!……0.1 0.2 0.50.6 0.2 0.1Example of Hidden Sequences For the yes/no recognizer, imagine we hear “yynooo” What are the scores of possible labelings?“y” “y” “n” “o” “o” “o”y y n o o o y y εεεs y y εεs s n n n o o o <s><s><s><s></s></s></s></s> ZEROV LowVV LowLow, but best?4The Viterbi Algorithm The Viterbi algorithm computes the best labeling for an observation sequence Incrementally computes best scores for subsequences Recurrence: Also store backtraces which record the argmaxesExample<s>yεsno</s>e0“<s>”e13“y”e27“n”e5“o”e5“o”e100“</s>”5Utilities So far: talked about beliefs Important difference between: Belief about some variables Rational action involving those variables Remember the midterm question? Next: utilitiesPreferences An agent chooses among: Prizes: A, B, etc. Lotteries: situations with uncertain prizes Notation:6Rational Preferences We want some constraints on preferences before we call them rational For example: an agent with intransitive preferences can be induced to give away all its money If B > C, then an agent with C would pay (say) 1 cent to get B If A > B, then an agent with B would pay (say) 1 cent to get A If C > A, then an agent with A would pay (say) 1 cent to get CRational Preferences Preferences of a rational agent must obey constraints. These constraints (plus one more) are the axioms of rationality Theorem: Rational preferences imply behavior describable as maximization of expected utility7MEU Principle Theorem: [Ramsey, 1931; von Neumann & Morgenstern, 1944] Given any preferences satisfying these constraints, there existsa real-valued function U such that: Maximum expected likelihood (MEU) principle: Choose the action that maximizes expected utility Note: an agent can be entirely rational (consistent with MEU) without ever representing or manipulating utilities and probabilities E.g., a lookup table for perfect tictactoeHuman Utilities Utilities map states to real numbers. Which numbers? Standard approach to assessment of human utilities: Compare a state A to a standard lottery Lpbetween ``best possible prize'' u+with probability p ``worst possible catastrophe'' u-with probability 1-p Adjust lottery probability p until A ~ Lp Resulting p is a utility in [0,1]8Utility Scales Normalized utilities: u+= 1.0, u-= 0.0 Micromorts: one-millionth chance of death, useful for paying to reduce product risks, etc. QALYs: quality-adjusted life years, useful for medical decisions involving substantial risk Note: behavior is invariant under positive linear transformation With deterministic prizes only (no lottery choices), only ordinal utilitycan be determined, i.e., total order on prizesMoney Money does not behave as a utility function Given a lottery L: Define expected monetary value EMV(L) Usually U(L) < U(EMV(L)) I.e., people are risk-averse Utility curve: for what probability pam I indifferent between: A prize x A lottery [p,$M; (1-p),$0] for large M? Typical empirical data, extrapolatedwith risk-prone behavior:9Example: Insurance Consider the lottery [0.5,$1000; 0.5,$0]? What is its expected monetary value? ($500) What is its certainty equivalent? Monetary value acceptable in lieu of lottery $400 for most people Difference of $100 is the insurance premium There’s an insurance industry because people will pay to reduce their risk If everyone were risk-prone, no insurance needed!Example: Human Rationality? Famous example of Allais (1953) A: [0.8,$4k; 0.2,$0] B: [1.0,$3k; 0.0,$0] C: [0.2,$4k; 0.8,$0] D: [0.25,$3; 0.75,$0] Most people prefer B > A, C > D But if U($0) = 0, then B > A ⇒ U($3k) > 0.8 U($4k) C > D ⇒ 0.8 U($4k) > U($3k)10Decision Networks Extended BNs Chance nodes (circles, like in BNs) Decision nodes (rectangles) Utility nodes (diamonds) Can query to find action with max expected utility Online applets if you want to play with theseValue of Information Idea: compute value of acquiring each possible piece of evidence Can be done directly from decision network Example: buying oil drilling rights Two blocks A and B, exactly one has oil, worth k Prior probabilities 0.5 each, mutually exclusive Current price of each block is k/2 ``Consultant'' offers accurate survey of A. Fair price? Solution: compute expected value of information= expected value of best action given the information minus expected value of best action without information Survey may say ``oil in A'' or ``no oil in A'', prob 0.5 each (given!)= [0.5 * value of ``buy A'' given ``oil in A'‘] +[0.5 * value of ``buy B'' given ``no oil in A'']–0= [0.5 * k/2] + [0.5 * k/2] - 0 = k/211General Formula Current evidence E, current best action α Possible action outcomes Si, potential new evidence Ej Suppose we knew Ej= ejk, then we would choose α(ejk) s.t. BUT Ejis a random variable whose value is currently unknown, so: Must compute expected gain over all possible values (VPI = value of perfect information)VPI Properties Nonnegative in expectation Nonadditive ---consider, e.g., obtaining Ejtwice
View Full Document
Unlocking...