Roadmap Embodied and Grounded Introduction and Overview Learning from uninterpreted experience A robot must be embodied Its sensorimotor system provides continuous interaction with an external environment Pierce Kuipers 1997 Abstraction of views actions and states Provost Kuipers Miikkulainen 2006 Its knowledge must be grounded Learning objects and actions Symbolic knowledge representation is directly or indirectly based on continuous sensorimotor interaction Modayil Kuipers 2004 2006 2007 1 2 Goal Reliable Predictions Bootstrap Learning Reliable predictions are impossible at the pixel level Starting with unknown sensors and effectors the learning agent must learn Too much detail to predict Need abstraction Search for a representation for states and actions such that the nature of its sensorimotor system and the structure of its environment Covers enough actions for useful behavior States are fine grained enough to express the prerequisites for reliable actions States are coarse grained enough to abstract away variation in action results The agent s goal is to find a representation for states and actions that supports reliable predictions about the results of actions and therefore to be able to make useful plans Intrinsic reward reliable predictions 3 4 One Method for Abstraction Distinctive States Discrete states and places are abstracted from patterns of continuous behavior A distinctive state location plus orientation is the isolated fixed point of a hill climbing control law x a x Hill climbing eliminates cumulative position error Reduces image variability due to pose variation Reliability state abstraction and feedback control 5 6 1 Reliable Actions The Spatial Semantic Hierarchy Reliable motion abstracts to a causal schema x a x A hierarchy of distinct ontologies x and x are distinctive states dstates Action a consists of trajectory following then hillclimbing leading reliably from x to x Control select control laws to move reliably among distinctive states Causal actions link states which have sensory views Topological places paths and regions linked by connectivity order containment Metrical frames of reference distance direction shape Between distinctive states actions are functionally deterministic x x How do we learn the foundation for this 7 8 Map Learning with Uninterpreted Sensors and Effectors A Thought Experiment Suppose a robot awakes with completely uninterpreted sensors and effectors Pierce Kuipers AIJ 1997 Simulated robot How can it learn a map of its world 24 sonar like range sensors One range sensor is broken constant Battery voltage sensor Digital compass Odometry Many aspects replicated with real sensors 9 Bootstrapping its way up Distinguish Sensor Groups Given sensor values s0 sn 1 for t in 0 T Compute distances d1 si sj and d2 si sj si sj if d1 si sj and d2 si sj are both small Learn the sensorimotor system 10 Distinguish the sensory modalities Learn the structure of each modality Identify a set of primitive actions Learn how actions affect sensory features small wrt both distributions si sj is the transitive closure of si sj Learn control laws Groups are the equivalence classes of si sj Learn homing hill climbing control laws Learn open loop trajectory following Learn closed loop trajectory following Separates range sensors from others Works pretty well on laser sonar IR bump Somewhat fragile Need softer clustering Learn a map of the environment SSH 11 12 2 Organization of each Sensor Array Estimate Sensor Similarities Locate s0 sk in k according to d1 si sj Use PCA to find dominant eigenvectors Start with a disorganized bag of pixels Determine pairwise sensor distances Project into low dimensional space Relax to best match d1 si sj N d1 si s j 1 N s t s t i k j k k 1 This is l1 distance Isomap uses l2 Euclidean distance Tenenbaum et al 2000 Olsson et al 2006 uses mutual information 13 Laser Rangefinder array 14 The Roving Eye The same method works applied to real data from the laser rangefinder array 180 rays 15 16 Likewise with low res vision Structure of the Roving Eye Low res retina with 15 x 15 225 pixels Compare to laser range finder with 180 scans 17 18 3 This is where space enters the ontology Organizing Visual Pixels Applied Isomap Tenenbaum et al 2000 to one minute of video Before this we had only time and change Correlation gives evidence of similarity Derive a mathematical similarity space Spatial structure reflects sensor anatomy The world is continuous almost everywhere Sensor elements are near each other Once we have space we can define motion 19 Find Primitive Actions Primitive Actions for the Roving Eye Given an intensity field S x y t define a motion vector field v 20 S S S S where S t x y Find primitive actions matching the principal eigenvectors of the average motion vector field 21 22 Local State Variables Identify Features Search for reliable relations between primitive actions and feature changes Then reliability conditional on context 23 24 4 Reliable Action Feature Relations Given a feature at egocentric r Define Homing Hill Climbing Control Laws u0 Turn reliably changes keeps r constant u1 Travel s if 0 reliably decreases r if 90 reliably keeps r constant if 180 reliably increases r 25 Define Open Loop TrajectoryFollowing Control Laws 26 Define Closed Loop TrajectoryFollowing Control Laws 27 28 Map Learning Given the ability to move reliably from distinctive state to distinctive state Build the causal map States actions views causal schemas etc Build the topological map Places paths regions connectivity etc Build the metrical map on its skeleton Join local frames of reference into a global frame The Spatial Semantic Hierarchy 29 30 5 A Strategy for Bootstrap Learning Research A Couple of Problems Devise a hierarchy of learning problems that build on each others results Find statistical learning methods for each one Avoid domain specific knowledge Try for biological plausibility Too much prior knowledge The feature generator embodies knowledge about continuous functions local min max etc Hill climbing and Trajectory following behaviors use PI and PD control templates Jefferson Provost s SODA solves these problems After finding a solution minimize the required statistical toolkit Distinguish evolutionary from individual learning 31 Roadmap 32 A Step Toward Learning Vision Introduction and Overview Learning from uninterpreted experience Learning the organization of retinal cells Pierce Kuipers 1997 The brain includes several
View Full Document
Unlocking...