1CS 2710 Foundations of AICS 2710 Foundations of AILecture 20-aMilos [email protected] Sennott SquareUtility theoryCS 2710 Foundations of AISelection based on expected values• Until now: The optimal action choice was the option that maximized the expected monetary value. • But is the expected monetary value always the quantity we want to optimize? Stock 1Stock 2Bank0.60.4110900.40.6140801011.01001.0Home102104101100(up)(down)(up)(down)2CS 2710 Foundations of AISelection based on expected values• Is the expected monetary value always the quantity we want to optimize? • Answer: Yes, but only if we are risk-neutral.• But what if we do not like the risk (we are risk-averse)?• In that case we may want to get the premium for undertaking the risk (of loosing the money)• Example: – we may prefer to get $101 for sure against $102 in expectation but with the risk of loosing the money• Problem: How to model decisions and account for the risk?• Solution: use utility function, and utility theoryCS 2710 Foundations of AIUtility function• Utility function (denoted U)– Quantifies how we “value” outcomes, i.e., it reflects our preferences – Can be also applied to “value” outcomes other than money and gains (e.g. utility of a patient being healthy, or ill)• Decision making:– uses expected utilities (denoted EU)the utility of outcome xImportant !!!• Under some conditions on preferences we can always design the utility function that fits our preferences)()()( xXUxXPXEUXx===∑Ω∈)( xXU =3CS 2710 Foundations of AIUtility theory• Defines axioms on preferences that involve uncertainty and ways to manipulate them. • Uncertainty is modeled through lotteries– Lottery: • Outcome A with probability p• Outcome C with probability (1-p)• The following six constraints are known as the axioms of utility theory. The axioms are the most obvious semantic constraints on preferences with lotteries. • Notation:- preferable- indifferent (equally preferable)]:)1(;:[ CpAp−f~CS 2710 Foundations of AIAxioms of the utility theory• Orderability: Given any two states, the a rational agent prefers one of them, else the two as equally preferable. • Transitivity: Given any three states, if an agent prefers A to B and prefers B to C, agent must prefer A to C. • Continuity: If some state B is between A and C in preference, then there is a p for which the rational agent will be indifferent between state B and the lottery in which A comes with probability p, C with probability (1-p).)~()()( BAABBA ∨∨ ff)()()( CACBBA fff ⇒∧BCpAppCBA ~]:)1(;:[)(−∃⇒ff4CS 2710 Foundations of AIAxioms of the utility theory• Substitutability: If an agent is indifferent between two lotteries, A and B, then there is a more complex lottery in which A can be substituted with B. • Monotonicity: If an agent prefers A to B, then the agent must prefer the lottery in which A occurs with a higher probability • Decomposability: Compound lotteries can be reduced to simpler lotteries using the laws of probability. ]:)1(;:[~]:)1(;:[)~( CpBpCpApBA−−⇒]):)1(;:[]:)1(;:[()( BqAqBpApqpBA −−⇔>⇒ ff]:)1)(1(;:)1(;:[]]:)1(;:[:)1(;:[CqpBqpApCqBqpAp−−−⇒−−CS 2710 Foundations of AIUtility theoryIf the agent obeys the axioms of the utility theory, then1. there exists a real valued function U such that:2. The utility of the lottery is the expected utility, that is the sum of utilities of outcomes weighted by their probability3. Rational agent makes the decisions in the presence of uncertainty by maximizing its expected utilityBABUAU f⇔> )()(BABUAU ~)()(⇔=)()1()(]:)1(;:[ BUpApUBpApU−+=−5CS 2710 Foundations of AIUtility functionsWe can design a utility function that fits our preferences if they satisfy the axioms of utility theory. • But how to design the utility function for monetary values so that they incorporate the risk?• What is the relation between utility function and monetary values?• Assume we loose or gain $1000. – Typically this difference is more significant for lower values (around $100 -1000) than for higher values (~ $1,000,000)• What is the relation between utilities and monetary value for a typical person?CS 2710 Foundations of AIUtility functions• What is the relation between utilities and monetary value for a typical person?• Concave function that flattens at higher monetary valuesutilityMonetary value100,0006CS 2710 Foundations of AIUtility functions• Expected utility of a sure outcome of 750 is 750utilityMonetary value1000500750EU(sure 750)U(x)CS 2710 Foundations of AIUtility functionsAssume a lottery L [0.5: 500, 0.5:1000]• Expected value of the lottery = 750• Expected utility of the lottery EU(L) is different:– EU(L) = 0.5U(500) + 0.5*U(1000)utilityMonetary value1000500750EU line for lotteries with outcomes 500 and 1000EU(lottery L)Lottery L: [0.5: 500, 0.5:1000] U(x)7CS 2710 Foundations of AIUtility functions• Expected utility of the lottery EU(lottery L) < EU(sure 750)• Risk aversion – a bonus is required for undertaking the risk utilityMonetary value1000500750EU(lottery L)EU(sure 750)Lottery L: [0.5: 500, 0.5:1000]
View Full Document
Unlocking...