Learning Reactive Behavior in Autonomous Vehicles: SAMUELSAMUELMotivation for SAMUELSlide 4SAMUEL (cont)Experiment Domain: Autonomous Underwater Vehicle navigation and collision avoidanceExperiment ResultsDomain: ROBOT Continuous and embedded learningContinuous and Embedded learning ModelExecution ModelLearning ModuleExperimentExperiment (cont)Slide 14Slide 15ReferencesLearning Reactive Behavior in Autonomous Vehicles:SAMUEL•Sanaa KamariSAMUEL•Computer system that learns reactive behavior for autonomous vehicles.–Reactive behavior is the set of actions taken by an AV as a reaction to sensor readings.•uses Genetic algorithm to improve decision making rules.•Each individual in SAMUEL is an entire rule set or strategy.Motivation for SAMUEL•Learning facilitates extraction of rules from the expert.•Rules are context based => impossible to account for every situation.–Given a set of conditions, the system is able to learn the rules of operation from observing and recording his own actions.•Samuel uses a simulation environment to learn.SAMUEL•Problem specific module.–The world model and its interface.–Set of internal and external sensors–Controllers that control the AV simulator–Critic component that criticizes the success or failure of the AV.[1]SAMUEL (cont)Performance module–Matches the rules.–Performs conflict resolution.–Assign some strength values to the rules. •Learning module.–Uses GA to develop reactive behavior, as a set of condition-reaction rules.•GA searches for the behavior to exhibit the best performance –Behaviors are evaluated in real world model.–Behaviors are selected for duplication and modification.[1]Experiment Domain: Autonomous Underwater Vehicle navigation and collision avoidance•Training the AUV simulator by virtually positioning it in the center of a field with 25 mines, and an objective outside the field.•2D AUV must navigate through a dense mine field toward a stationary object.•AUV Actions: set speed and direction each decision cycle. •System does not learn path, but a set of rules that reactively decide a move at each step.Experiment Results•Great improvement in both static and moving mines.•SAMUEL shows that reactive behavior can be learned.[1]Domain: ROBOT Continuous and embedded learning•To create Autonomous systems that continue to learn throughout their lives.•To adapt a robot’s behavior in response to changes in its operating environment and capabilities.•experiment: robot learns to adapt to failure in its sonar sensors.Continuous and Embedded learning Model•Execution module: controls the robot’s interaction with its environment.•Learning module: continuously tests new strategies for the robot against a simulation model of its environment.[2]Execution Model•Includes a rule-based system that operates on reactive (stimulus-response) rules.–IF range = [35, 45] AND front sonar < 20 AND right sonar > 50 THEN SET turn = -24 (Strength 0.8)•Monitor: Identifies symptoms of sonar failure.–measures output of sonar, compare it to recent readings and direction of motion.–Modifies simulation used by learning sys to replicate failure.Learning Module•Uses SAMUEL: uses Genetic algorithm to improve decision making rules.Experiment•Task requires Robot to go from one side of a room to the other through an opening.•Robot placed randomly 4 ft from back wall.•Location of opening is random.•Center of front wall is 12.5ft from back wallExperiment (cont)•Robot begins with a set of default rules for moving toward the goal.•Learning starts with simulation that includes and all sonars working.•After an initial period one ore more sonars are blinded.•Monitor detects failed sonars, learning simulation is adjusted to reflect failure.•Population of competing strategies is re-initialized and learning continues.•The online Robot uses the best rules discovered by the learning system since the last change to the learning simulation model,Experiment Results•Robot in motion with all sensors intact:–a) during run and b) at goal.•Robot in motion after adapting to loss of three sensors: front, front right and right:–a) during run, and b) at goal.[2]Experiment Results•a) Robot with full sensors passing directly through doorway. •b) Robot with front sonar covered.•c) Robot after adapting to covered sonar. It uses side sonar to find opening, and then turns into the opening.[2]References•[1]. A. C. Schultz and J. J.Grefenstetts, “Using a genetic algorithm to learn reactive behavior for autonomous vehicles,” in Proceedings of the AIAA Guidance, Navigation, and Control Conference, (Hilton Head, SC), 1992.•[2]. A. C. Schultz and J. J.Grefenstetts, ”Continuous and Embedded Learning in Autonomous Vehicles: Adapting to Sensor Failures”, in Proceeding of SPIE vol. 4024, pg 55-62,