CS 416 Artificial Intelligence Lecture Lecture 22 Agents Agents Chess Article Deep Deep Blue Blue IBM IBM 418 418 processors processors 200 200 million million positions positions per per second second Deep Deep Junior Junior Israeli Israeli Co Co 88 processors processors 33 million million positions positions per per second second Kasparov Kasparov 100 100 billion billion neurons neurons in in brain brain 22 moves moves per per second second But But there there are are 85 85 billion billion ways ways to to play play the the first first four four moves moves Chess Article 1997 1997 Kasparov Kasparov Lost Lost to to Deep Deep Blue Blue 2002 2002 Kramnik Kramnik tied tied Deep Deep Junior Junior current current World World Champion Champion 2003 2003 Kasparov Kasparov current current number number 1 1 plays plays Deep Deep Junior Junior Jan Jan 26 26 Feb Feb 77 Chess Article Cognitive Cognitive psychologists psychologists report report chess chess is is aa game game of of pattern pattern matching matching for for humans humans But But what what patterns patterns do do we we see see What What rules rules do do we we use use to to evaluate evaluate perceived perceived patterns patterns What is an agent Perception Perception Sensors Sensors receive receive input input from from environment environment Keyboard Keyboard clicks clicks Camera Camera data data Bump Bump sensor sensor Action Action Actuators Actuators impact impact the the environment environment Move Move aa robotic robotic arm arm Generate Generate output output for for computer computer display display Perception Percept Percept Perceptual Perceptual inputs inputs at at an an instant instant May May include include perception perception of of internal internal state state Percept Percept Sequence Sequence Complete Complete history history of of all all prior prior percepts percepts Do Do you you need need aa percept percept sequence sequence to to play play Chess Chess An agent as a function Agent Agent maps maps percept percept sequence sequence to to action action f ps a ps p Agent Agent Set Set of of all all inputs inputs known known as as state state space space Agent Agent Function Function IfIf inputs inputs are are finite finite aa table table can can store store mapping mapping Scalable Scalable Reverse Reverse Engineering Engineering Evaluating agent programs We We agree agree on on what what an an agent agent must must do do Can Can we we evaluate evaluate its its quality quality Performance Performance Metrics Metrics Very Very Important Important Frequently Frequently the the hardest hardest part part of of the the research research problem problem Design Design these these to to suit suit what what you you really really want want to to happen happen Rational Agent For For each each percept percept sequence sequence aa rational rational agent agent should should select select an an action action that that maximizes maximizes its its performance performance measure measure Example Example autonomous autonomous vacuum vacuum cleaner cleaner What What is is the the performance performance measure measure Penalty Penalty for for eating eating the the cat cat How How much much Penalty Penalty for for missing missing aa spot spot Reward Reward for for speed speed Reward Reward for for conserving conserving power power Learning and Autonomy Learning Learning To To update update the the agent agent function function in in light light of of observed observed performance performance of of percept sequence percept sequence to to action action pairs pairs Explore Explore new new parts parts of of state state space space Learn Learn from from trial trial and and error error Change Change internal internal variables variables that that influence influence action action selection selection Adding intelligence to agent function At At design design time time Some Some agents agents are are designed designed with with clear clear procedure procedure to to improve improve performance performance over over time time Really Really the the engineer s engineer s intelligence intelligence Camera based Camera based user user identification identification At At run time run time Agent Agent executes executes complicated complicated equation equation to to map map input input to to output output Between Between trials trials With With experience experience agent agent changes changes its its program program parameters parameters How big is your percept Dung Dung Beetle Beetle Largely Largely feed feed forward forward Sphex Sphex Wasp Wasp Reacts Reacts to to environment environment feedback feedback but but not not learning learning A A Dog Dog Reacts Reacts to to environment environment and and can can significantly significantly alter alter behavior behavior Qualities of a task environment Fully Fully Observable Observable Agent Agent need need not not store store any any aspects aspects of of state state The The Brady Brady Bunch Bunch as as intelligent intelligent agents agents Volume Volume of of observables observables may may be be overwhelming overwhelming Partially Partially Observable Observable Some Some data data is is unavailable unavailable Maze Maze Noisy Noisy sensors sensors Qualities of a task environment Deterministic Deterministic Always Always the the same same outcome outcome for for state action state action pair pair Stochastic Stochastic Not Not always always predictable predictable random random Partially Partially Observable Observable vs vs Stochastic Stochastic My My cats cats think think the the world world is is stochastic stochastic Physicists Physicists think think the the world world is is deterministic deterministic Qualities of a task environment Markovian Markovian Future Future state state only only depends depends on on current current state state Episodic Episodic Percept Percept sequence sequence can can be be segmented segmented into into independent independent temporal temporal categories categories Behavior Behavior at at traffic traffic light light independent independent of of previous previous traffic traffic Sequential Sequential Current Current decision decision could could affect affect all all future future decisions decisions Which Which is is easiest easiest to to program program Qualities of a task environment Static Static Environment Environment doesn t doesn t change change over over time time Crossword Crossword puzzle puzzle Dynamic Dynamic Environment Environment changes