Annual Conference of the Prognostics and Health Management Society, 2010 1 Model-Based Assurance of Diagnostic Procedures for Complex Systems Tolga Kurtoglu1, Robyn Lutz2, and Martin S. Feather3 1 Mission Critical Technologies @ NASA Ames Research Center, Moffett Field, CA, 94035, USA [email protected] 2 Jet Propulsion Laboratory/Caltech, Pasadena, CA, 91109, and Iowa State University, USA [email protected] 3 Jet Propulsion Laboratory, California Institute of Technology, CA, 91109, USA [email protected] ABSTRACT Verifying diagnostic procedures for complex systems is hard and labor-intensive. Usually this verification is accomplished primarily through extensive review of the procedures by experts. We aim to augment this review process by using insights from comparing the diagnostic steps described in the procedural definitions with diagnostics information derived from existing models of the system. These comparisons offer various conformance checks between the manually developed diagnostic procedures and the diagnostic trees auto-generated from the diagnostic system models. We previously described our DTV (Diagnostic Tree for Verification) technique based on these comparisons. This paper describes an extension to DTV, and reports results of an application of DTV to a representative system’s diagnostic procedures. Specifically, it outlines four analyses (branch analysis, root cause coverage, path verification, and efficiency) that can be performed using DTV; illustrates the process for applying DTV; and reports results from our application of DTV to assure fifteen of the procedures developed for diagnosing problems in an electrical power system testbed for spacecraft. 1 INTRODUCTION The operation of complex engineered systems requires the development of diagnostic procedures. These provide a detailed set of instructions to help operators monitor the system’s parameters and respond to potential problems by detecting and identifying faults, and guides them in performing system reconfiguration or restoration. These procedural definitions include a complicated mix of software checks and calibrations, conditional commands, manual inputs, checks of console data, and inspection of physical equipment. The crew of the Space Shuttle, for example, relies on a collection of procedural definitions and checklists in order to interpret any potential anomalies, figure out the root cause of problems, and work on mitigating the root-cause failure (Hayashi et al., 2008). As a result, crew safety and mission success become highly dependent on the correctness of the diagnostic procedures. It is therefore imperative that these procedures are verified before being used. However, verifying diagnostic procedures for complex systems is hard and labor-intensive. Usually this verification is heavily dependent on extensive review of the procedures by experts. In this research, we aim to augment this review process by using insights from comparing the diagnostic steps described in the procedural definitions with diagnostics information derived from existing models of the system. These comparisons offer various conformance checks between the manually developed diagnostic procedures and the diagnostic system models. Checking conformance in this way has two advantages. First, the models offer an independent perspective distinct from expert review. Second, there exists computer software that is able to systematically explore the diagnostic implications of a model, a task that can be quite intricate as systems grow in size and complexity. Both these advantages increase the likelihood of revealing errors (if present) in the diagnostic procedures. This approach can therefore contribute to assuring the correctness of diagnostic procedures for complex systems. Furthermore, this approach can also identify opportunities for improving the efficiency of diagnostic procedures by revealing theAnnual Conference of the Prognostics and Health Management Society, 2010 2 presence of redundant steps in the existing procedures, and/or by offering alternately structured procedures that are capable of arriving at the same diagnostic conclusions but in fewer steps. In previous work we introduced the DTV (Diagnostic Tree for Verification) technique (Kurtoglu et al., 2009). The key idea of DTV is to compare the text-based procedures for diagnosing faults during a system’s operations with the diagnostic trees auto-generated from a model of the system. This paper describes our investigation of, and an extension to, the DTV technique. The extension addresses the challenge of assuring the correctness of a text-based procedure in the (frequent) case when the auto-generated diagnostic tree uses different sequences of tests to arrive at its diagnostic conclusions. In such cases, if both trees are able to diagnose to the same sets of root causes, this offers some assurance, but does not guarantee, that the paths followed in the text-based procedure in fact imply their diagnostic conclusions. In our extension, we use the steps in a path of the text-based procedure to “drive” the system model so that once the end of the path is reached we can compare the path’s diagnosis with the model’s conclusion. The specific contributions of this paper over our previous work are: (1) describes four analyses (branch analysis, root cause coverage, path verification, and efficiency) that can be performed using DTV; (2) describes and illustrates the process for applying DTV; and (3) reports results from our application of DTV to assure fifteen of the procedures developed for diagnosing problems in an electrical power system testbed for spacecraft. In what follows, we first present a review of related work in verification of operational procedures. Section 3 describes the Diagnostic Tree for Verification method. Section 4 introduces the Electrical Power System Testbed in the ADAPT Lab at NASA Ames Research Center that is used as a case example in this study. Section 5