Pomerleau, D.A. (in press) Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving. To appear in Robot Learning, J. Connell and S. Mahadevan (eds.), Kluwer Academic Publishing. Knowledge-based Training of Artificial Neural Networks for Autonomous Robot Driving Dean A. Pomerleau Carnegie Mellon University School of Computer Science Pittsburgh, PA 15213-3890 Abstract Many real world problems quire a degree of flexibility that is diflicult to achieve using hand programmed algorithms. One such domain is vision-based autonomous driving. In this task, the dual challenges of a constantly changing environment coupled with a real the processing constrain make the flexibility and efficiency of a machine learning system essential. This chapter describes just such a learning system, called ALVINN (Autonomous Land Vehicle In a Neural Network). It presents the neural network architecture and training techniques that allow &VI" to drive in a variety of circumstanm including singlelane paved and unpaved roads, multilane lined and unlined roads, and obstacle-ridden on- and off- road environments, at speeds of up to 55 miles per hour. 1 Introduction Autonomous navigation is a difficult problem for traditional vision and robotic techniques, primarily because of the noise and variability associ- ated with real world scenes. Autonomous navigation systems based on traditional image processing and pattern recognition techniques often per- form well under certain conditions but have problems with others. Part of the difficulty stems from the fact that the processing performed by these systems remains fixed across various environments. Artificial neural networks have displayed promising performance and flexibility in other domains characterized by high degrees of noise and vari-Figure 1 : The CMU Navlab Autonomous Navigation Testbed ability, such as handwritten character recognition [6] and speech recognition[ 151 and face recognition[2]. ALVINN (Autonomous Land Vehicle In a Neural Network) is a system that brings the flexibility of connectionist learn- ing techniques to the task of autonomous robot navigation. Specifically, ALVINN is an artificial neural network designed to control the Navlab, the Carnegie Mellon autonomous driving test vehicle (See Figure 1). This chapter describes the architecture, training and performance of the ALVINN system. It demonstrates how simple connectionist networks can learn to precisely guide a mobile robot in a wide variety of situations when trained appropriately. In particular, this chapter presents training techniques that allow ALW" to learn in under 5 minutes to autonomously control the Navlab by watching a human driver's response tonew situations. Using these techniques, ALVINN has been trained to drive in a variety of circumstances including single-lane paved and unpaved roads, multilane lined and unlined roads, and obstacle-ridden on- and off-road environments, at speeds of up to 55 miles per hour.I I Figure 2: Neural network architecture for autonomous driving. 2 Network Architecture The basic network architecture employed ic the ALM" pystem is a single hidden layer feedforward neural network 8 See Figure 2 'he input layer now consists of a single30x32 unit "retina" onto which a -or image from either a video camera or a scanning laser rangefinder is p:ujccted. Each of the 960 input units is fully connected to the hidden layer of 4 units, which is in turn fully connected to the output layer. The 30 unit output layer is a linear representation of the cumntly appropriate steering direction which may serve to keep the vehicle on the mad or to prevent it from colliding with nearby obstacles'. The centexmost output unit represents the "travel straight ahead" condition, while units to the left and right of center represent successively sharper left and right turns. The units on the extreme left and right of the output vector represent turns with a 20m radius to the left and right respectively, and the units in between represent turns which decrease linearly in their curvature down to the "straight ahead" middle unit in the output vector. To drive the Navlab, an image from the appropriate sensor is reduced to 30x32 pixels and projected onto the input layer. After propagating activa- 'The task a particular driving network performs depends on the type of input sensor image and the driving situation it has been trained to handle.tion through the network, the output layer’s activation profile is translated into a vehicle steering command. The steering direction dictated by the network is taken to be the center of mass of the “hill” of activation sur- rounding the output unit with the highest activation level. Using the center of mass of activation instead of the most active output unit when detexmin- ing the direction to steer pennits finer steering corrections, thus improving ALVINN’s driving accuracy. 3 Network Training The network is trained to produce the comt steering direction using the backpropagation learning algorithm [ 131. In backpropagation, the network is first presented with an input and activation is propagated forward through the network to determine the network’s response. The network’s response is then compared with the known correct response. If the network’s actual response does not match the correct response, the weights between con- nections in the network are modified slightly to produce a response more closely matching the correct response. Autonomous driving has the potential to be an ideal domain for a supervised leaming algorithm like backpropagation since there is a readily available teaching signal or “correct response” in the form of the human driver’s current steering direction. In theory it should be possible to teach a network to imitate a person as they drive using the current sensor image as input and the person’s current steering direction as the desired output. This idea of training “on-the-fly’’ is depicted in Figure 3. Training on real images would dramatically reduce the human effort required to develop networks for new situations, by eliminating the need for a hand-programmed training example generator. On-the-fly training should also allow the system to adapt quickly to new situations. 3.1 Potential Problems There are two potential problems associated with training a network using live sensor images as a person drives. First, since the person steers the