Mel Siegel <[email protected]> / The Robotics Institute / Carnegie Mellon University /Pittsburgh PA 15213 USA / 412 268 8802 / FAX 412 268 5569on the web at http://www.cs.cmu.edu/~mws, click on link “FAST-TRACK route to my Lab’s publications”1998 IEEE International Workshop on Emerging Technologies, Intelligent Measurements and Virtual Systems for Instrumentation and Measurements(ETIMVIS’98), Minnesota Club, St. Paul, MN, USA - May 15-16, 1998Remote Enhanced Visual Inspection of Aircraft by a Mobile Robot, page 1 of 10 pages/usr10/mws/mss/etimvis98/text/etimvis98t-final.mkr as of April 16, 1998 2:12 pm1998 IEEE Workshop on Emerging Technologies, Intelligent Measurementand Virtual Systems for Instrumentation and Measurement - ETIMVIS’98St. Paul, MN, USA, May 15-16, 1998Remote Enhanced Visual Inspection of Aircraft by a Mobile RobotMel Siegel and Priyan GunatilakeThe Robotics Institute / School of Computer Science / Carnegie Mellon UniversityPittsburgh PA 15213 USAphone: +412 268 8802, fax: +412 268 5569, email: [email protected] skin inspection methodology, and the technical and economic advantages that can beexpected from doing it with the aid of robotics, automation, and computer vision technologies, arereviewed. Robots for deployment of inspection systems are described briefly. Computer visionmethods and algorithms for detection of cracks, surface corrosion, and subsurface corrosion arethen discussed in detail. Cracks and surface corrosion are both detected by algorithms thatpipeline preprocessing/enhancement, multiresolution/wavelet based feature extraction, andneural net based feature vector classification; subsurface corrosion is detected by a structuredlaser light surface profiling technique for the pillowing that subsurface corrosion causes.Functionality is illustrated by application to current data.I. IntroductionAircraft skins are regularly inspected both for known classes of problems (primarily cracking andcorrosion) in specific locations (known via prior experience and theoretical stress modeling), and forrandom damage, as from lightning strikes, bumps by ground support vehicles, etc. Approximately 90% ofthis inspection is visual; the remaining 10% is instrumented, primarily using eddy current sensors, withultrasonic and other instruments employed occasionally.Robotics and automation technology has potentially large contributions to make in this field: theadvantages of “getting the man off the airplane”, the tireless concentration of a robot, the guarantee ofproper and consistent instrument deployment protocol, the inherent navigational and record-keepingcapabilities of a mobile robot under computer control, etc., are all clear. Furthermore, in an environmentwhere the records that are currently preserved usually indicate only pass/fail, there are enormousopportunities for predictive maintenance that are enabled by bringing robotics, automation, data storage,and statistics into the picture.1,2We have built two mobile robots for this environment: ANDI, the Automated NonDestructiveInspector3, and CIMP, the Crown Inspection Mobile Platform. ANDI was designed to deploy eddy currentsensors; however inasmuch as ANDI employed suction cups to give it access to a large fraction of theaircraft skin surface, the lion's share of this project's effort went into mobility-related issues. To minimizethe distraction of building a machine with a comprehensive mobility capability, CIMP's motion wasrestricted to the aircraft crown; this allowed us to focus successfully on measurement issues. Because 90%of aircraft inspection in the commercial sector is visual, the economic viability of a robotic solution isdictated by its ability to enable remote inspection. An additional edge may be provided by enhanced visualinspection capabilities, e.g., image processing to aid the inspector in finding anomalies, and automatedimage understanding to bring anomalies to the inspector's attention. We will begin this paper bysummarizing the above background and context; these topics were presented in detail at ETVSIM’971. Wewill then present the results of our field trials of CIMP, providing aircraft inspectors with an interface toteleoperate it and to observe remote stereoscopic video with remote control of camera viewpoint and skinillumination angle and direction.In the main body of this paper we will show current laboratory progress toward image enhancementand automated image understanding, including algorithms that automatically mark cracks around rivetheads, patches of corroded skin surface, and skin surface deformations indicative of underlying subsurfacecorrosion.Mel Siegel <[email protected]> / The Robotics Institute / Carnegie Mellon University /Pittsburgh PA 15213 USA / 412 268 8802 / FAX 412 268 5569on the web at http://www.cs.cmu.edu/~mws, click on link “FAST-TRACK route to my Lab’s publications”1998 IEEE International Workshop on Emerging Technologies, Intelligent Measurements and Virtual Systems for Instrumentation and Measurements(ETIMVIS’98), Minnesota Club, St. Paul, MN, USA - May 15-16, 1998Remote Enhanced Visual Inspection of Aircraft by a Mobile Robot, page 2 of 10 pages/usr10/mws/mss/etimvis98/text/etimvis98t-final.mkr as of April 16, 1998 2:12 pmII. Background: skin defect detection and classificationTo our surprise and delight, aircraft inspectors have been spontaneous and enthusiastic advocates forusing computer image enhancement and automated image understanding for flaw detection; they are,however, skeptical about the likelihood that we will succeed at the latter.The goal of an image understanding algorithm for aircraft inspection is to recognize and classifycertain surface flaws that might appear in the live imagery. The recognition capability of an algorithm isachieved by correlating features of the imagery with prior or learned knowledge of the surface flaw types.However, developing a successful image understanding algorithm remains a non-trivial challenge, dueprimarily to the difficulty of generalizing and encoding in an algorithm the notions that humans use todiscriminate normal from defective, the limited resolution and dynamic range of practical imagingsystems, and the confounding effects of environment factors such as illumination.Given these limitations, an attractive scenario for application of image understanding algorithms inremote visual inspection is screening large volumes of image data. The image understanding algorithm canconservatively label