11Roadmap• Introduction and Overview• Learning from uninterpreted experience– Pierce & Kuipers, 1997• Abstraction of views, actions, and states– Provost, Kuipers & Miikkulainen, 2006• Learning objects and actions– Modayil & Kuipers, 2004, 2006, 20072Embodied and Grounded• A robot must be embodied– Its sensorimotor system provides continuousinteraction with an external environment.• Its knowledge must be grounded– Symbolic knowledge representation is(directly or indirectly) based on continuoussensorimotor interaction.3Bootstrap Learning• Starting with unknown sensors andeffectors, the learning agent must learn– the nature of its sensorimotor system, and– the structure of its environment.• The agent’s goal is to– find a representation for states and actions– that supports reliable predictions about theresults of actions,– and therefore to be able to make useful plans.4Goal: Reliable Predictions• Reliable predictions are impossible at the“pixel level”.– Too much detail to predict. Need abstraction.• Search for a representation for states andactions such that– Covers enough actions for useful behavior– States are fine-grained enough to express theprerequisites for reliable actions– States are coarse-grained enough to abstractaway variation in action results• Intrinsic reward: reliable predictions.5One Method for Abstraction• Discrete states and places are abstractedfrom patterns of continuous behavior.6Distinctive States• A distinctive state (location plus orientation) is theisolated fixed-point of a hill-climbing control law.– Hill-climbing eliminates cumulative position error.– Reduces image variability due to pose variation.• Reliability: state abstraction and feedback controlxx’a27Reliable Actions• Reliable motion abstracts to a causal schema<x,a,x’>– x and x’ are distinctive states (dstates),– Action a consists of trajectory-following then hill-climbing, leading reliably from x to x’.• Between distinctive states, actions are functionallydeterministic.xx’8The Spatial Semantic HierarchyA hierarchy of distinct ontologies:– Control: select control laws to move reliablyamong distinctive states.– Causal: actions link states, which havesensory views.– Topological: places, paths, and regions linkedby connectivity, order, containment.– Metrical: frames of reference, distance,direction, shape.• How do we learn the foundation for this?9A Thought Experiment• Suppose a robot awakes with completelyuninterpreted sensors and effectors.• How can it learn a map of its world?10Map-Learning withUninterpreted Sensors and Effectors• [Pierce & Kuipers, AIJ, 1997]• Simulated robot– 24 sonar-like range sensors– One range sensor is broken (constant)– Battery voltage sensor– Digital compass– Odometry• Many aspects replicated with real sensors.11Bootstrapping its way up• Learn the sensorimotor system– Distinguish the sensory modalities– Learn the structure of each modality– Identify a set of primitive actions– Learn how actions affect sensory features• Learn control laws– Learn homing (hill-climbing) control laws– Learn open-loop trajectory-following– Learn closed-loop trajectory-following• Learn a map of the environment (SSH)12Distinguish Sensor Groups• Given sensor values s0 … sn-1 for t in [0,T]• Compute distances d1(si,sj) and d2(si,sj)• si ≈ sj if d1(si,sj) and d2(si,sj) are both small– “small” wrt both distributions• si ~ sj is the transitive closure of si ≈ sj– Groups are the equivalence classes of si ~ sj• Separates range sensors from others– Works pretty well on laser/sonar/IR/bump– Somewhat fragile. Need softer clustering.313Estimate Sensor Similarities• Start with a disorganized “bag of pixels”• Determine pairwise sensor distances• This is l1 distance.• Isomap uses l2 (Euclidean) distance.– [Tenenbaum, et al, 2000]• Olsson, et al [2006] uses mutual information.! d1(si,sj) =1N| sik=1N"(tk) # sj(tk) |14Organization of each Sensor Array• Locate s0 … sk in ℜk according to d1(si,sj)• Use PCA to find dominant eigenvectors– Project into low-dimensional space– Relax to best match d1(si,sj)15Laser Rangefinder array• The same method works, applied to real datafrom the laser rangefinder array (180 rays)16The “Roving Eye”17Structure of the “Roving Eye”18Likewise with low-res vision• Low-res retina with 15 x 15 = 225 pixels– Compare to laser range-finder with 180 scans419Organizing Visual Pixels• Applied Isomap [Tenenbaum, et al, 2000]to one minute of video.20This is wherespace enters the ontology• Before this, we had only time and change.– Correlation gives evidence of similarity– Derive a mathematical similarity space• Spatial structure reflects sensor anatomy– The world is continuous almost everywhere– Sensor elements are near each other• Once we have space, we can define motion.21Find Primitive Actions• Given an intensity field S(x,y,t), define amotion vector field• Find primitive actions matching the principaleigenvectors of the average motion vectorfield! v = "#S#t$S where $S =#S#x,#S#y% & ' ( ) * 22Primitive Actions for the“Roving Eye”23IdentifyFeatures24Local State Variables• Search for reliable relations betweenprimitive actions and feature changes• Then reliability conditional on context.525Reliable Action-Feature Relations• Given a feature at egocentric (r,φ).• u0 = Turn Δθ– reliably changes φ, keeps r constant.• u1 = Travel Δs– if φ = 0, reliably decreases r– if φ = ±90, reliably keeps r constant– if φ = 180, reliably increases r26Define Homing (Hill-Climbing)Control Laws27Define Open-Loop Trajectory-Following Control Laws28Define Closed-Loop Trajectory-Following Control Laws2930Map Learning• Given the ability to move reliably fromdistinctive state to distinctive state• Build the causal map– States, actions, views, causal schemas, etc.• Build the topological map– Places, paths, regions, connectivity, etc.• Build the metrical map on its skeleton– Join local frames of reference into a global frame• The Spatial Semantic Hierarchy631A Strategy forBootstrap Learning Research• Devise a hierarchy of learning problems thatbuild on each others’ results.• Find statistical learning methods for each one– Avoid domain-specific knowledge– Try for biological plausibility• After finding a solution, minimize therequired
View Full Document
Unlocking...