Page 1CS 287: Advanced RoboticsFall 2009Lecture 1: IntroductionPieter AbbeelUC Berkeley EECS http://www.cs.berkeley.edu/~pabbeel/cs287-fa09 Instructor: Pieter Abbeel Lectures: Tuesdays and Thursdays, 12:30pm-2:00pm, 405 Soda Hall Office Hours: Thursdays 2:00-3:00pm, and by email arrangement. In 746 Sutardja Dai HallwwwPage 2 Communication: Announcements: webpage Email: [email protected] Office hours: Thursday 2-3pm + by email arrangement, 746 SDH Enrollment: Undergrads stay after lecture and see meAnnouncements Prerequisites: Familiarity with mathematical proofs, probability, algorithms, linear algebra, calculus. Ability to implement algorithmic ideas in code. Strong interest in robotics Work and grading Four large assignments (4 * 15%) One smaller assignment (5%) Open-ended final project (35%) Collaboration policy: Students may discuss assignments with each other. However, each student must code up their solutions independently and write down their answers independently. Class DetailsPage 3 Learn the issues and techniques underneath state of the art robotic systems Build and experiment with some of the prevalent algorithms Be able to understand research papers in the field Main conferences: ICRA, IROS, RSS, ISER, ISRR Main journals: IJRR, T-RO, Autonomous Robots Try out some ideas / extensions of your ownClass Goals Logistics --- questions? [textbook slide forthcoming] A few sample robotic success stories Outline of topics to be coveredLecture outlinePage 4 Darpa Grand Challenge First long-distance driverless car competition 2004: CMU vehicle drove 7.36 out of 150 miles 2005: 5 teams finished, Stanford team won Darpa Urban Challenge (2007) Urban environment: other vehicles present 6 teams finished (CMU won) Ernst Dickmanns / Mercedes Benz: autonomous car on European highways Human in car for interventions Paris highway and 1758km trip Munich -> Odense, lane changes at up to 140km/h; longest autonomous stretch: 158kmDriverless carsKalman filtering, Lyapunov, LQR, mapping, (terrain & object recognition)Autonomous Helicopter Flight[Coates, Abbeel & Ng]Kalman filtering, model-predictive control, LQR, system ID, trajectory learningPage 5Four-legged locomotioninverse reinforcement learning, hierarchical RL, value iteration, receding horizon control, motion planning[Kolter, Abbeel & Ng]Two-legged locomotion[Tedrake +al.]TD learning, policy search, Poincare map, stabilityPage 6Mapping“baseline” : Raw odometry data + laser range finder scans[Video from W. Burgard and D. Haehnel]MappingFastSLAM: particle filter + occupancy grid mapping[Video from W. Burgard and D. Haehnel]Page 7Mobile ManipulationSLAM, localization, motion planning for navigation and grasping, grasp point selection, (visual category recognition, speech recognition and synthesis)[Quigley, Gould, Saxena, Ng + al.] Control: underactuation, controllability, Lyapunov, dynamic programming, LQR, feedback linearization, MPC Estimation: Bayes filters, KF, EKF, UKF, particle filter, occupancy grid mapping, EKF slam, GraphSLAM, SEIF, FastSLAM Manipulation and grasping: force closure, grasp point selection, visual servo-ing, more sub-topics tbd Reinforcement learning: value iteration, policy iteration, linear programming, Q learning, TD, value function approximation, Sarsa, LSTD, LSPI, policy gradient, inverse reinforcement learning, reward shaping, hierarchical reinforcement learning, inference based methods, exploration vs. exploitation Brief coverage of: system identification, simulation, pomdps, k-armed bandits, separation principle Case studies: autonomous helicopter, Darpa Grand/Urban Challenge, walking, mobile manipulation. Outline of TopicsPage 8 Overarching theme: mathematically capture What makes control problems hard What techniques do we have available to tackle the hard problems E.g.: “Helicopters have underactuated, non-minimum phase, highly non-linear and stochastic (within our modeling capabilities) dynamics.” Hard or easy to control?1. Control Under-actuated vs. fully actuated Example: acrobot swing-up and balance task1. Control (ctd)Page 9 Other mathematical formalizations of what makes some control problems easy/hard: Linear vs. non-linear Minimum-phase vs. non-minimum phase Deterministic vs. stochastic Solution and proof techniques we will study: Lyapunov, dynamic programming, LQR, feedback linearization, MPC1. Control (ctd) Bayes filters: KF, EKF, UKF, particle filter One of the key estimation problems in robotics: Simultaneous Localization And Mapping (SLAM) Essence: compute posterior over robot pose(s) and environment map given (i) Sensor model (ii) Robot motion model Challenge: Computationally impractical to compute exact posterior because this is a very high-dimensional distribution to represent [You will benefit from 281A for this part of the course.]2. EstimationPage 10 Extensive mathematical theory on grasping: force closure, types of contact, robustness of grasp Empirical studies showcasing the relatively small vocabulary of grasps being used by humans (compared to the number of degrees of freedom in the human hand) Perception: grasp point detection3. Grasping and Manipulation Learning to act, often in discrete state spaces value iteration, policy iteration, linear programming, Q learning, TD, value function approximation, Sarsa, LSTD, LSPI, policy gradient, inverse reinforcement learning, reward shaping, hierarchical reinforcement learning, inference based methods, exploration vs. exploitation4. Reinforcement learningPage 11 system identification: frequency domain vs. time domain Simulation / FEM Pomdps k-armed bandits separation principle …5. Misc. Topics Control Tedrake lecture notes 6.832: https://svn.csail.mit.edu/russt_public/6.832/underactuated.pdf Estimation Probabilistic Robotics, Thrun, Burgard and Fox. Manipulation and grasping - Reinforcement learning Sutton and Barto, Reinforcement Learning (free online) Misc. topics -Reading materialsPage 12 Next lecture we will start with our study of
View Full Document