Leading Indicators for Systems Engineering Effectiveness Presentation for NDIA SE Conference October 28, 2009Growing Interest in SE EffectivenessBackground of the Systems Engineering Leading Indicators ProjectA Collaborative Industry EffortObjectives of the projectSE Leading Indicator DefinitionLeading IndicatorsInteractions Among FactorsCriteria of Leading IndicatorsSystems Engineering Leading IndicatorsList of IndicatorsFields of Information Collected for Each IndicatorSlide 13Example of Section 3 ContentsExample of Section 4 ContentsExample of Section 4 Contents (Cont’d)Slide 17Indicator’s Usefulness for Gaining Insight to the Effectiveness of Systems Engineering (1 of 3)Indicator’s Usefulness for Gaining Insight to the Effectiveness of Systems Engineering (2 of 3)Looking Forward – What Next?Next Steps/Action ItemsNew IndicatorsAdditional Information on Specific Application and RelationshipsSELI versus SE Activities of ISO/IEC 15288NAVAIR Applied Leading Indicators (ALI) MethodologyInteraction with SERC SE Effectiveness Measurement ProjectQUESTIONS?11 Leading Indicators for Systems Engineering EffectivenessPresentation for NDIA SE ConferenceOctober 28, 2009Garry Roedler Lockheed Martin SY STEMS ENGINEERING LEADING INDICAT ORS GUIDE Version 1.0 J une 15, 2007 Supersedes Beta Release, December 2005 Editors Garry R oedler Lockheed Martin Corporation [email protected] Donna H. R ho des Massachusetts Institute of Technology [email protected] Developed and P ublis hed by Members of INCOSE Technical Product Number: INCOSE-TP-2005-001-0222Growing Interest in SE Effectiveness•Questions about the effectiveness of the SE Questions about the effectiveness of the SE processes and activities are being askedprocesses and activities are being asked–DoDDoD–INCOSEINCOSE–OthersOthers•Key activities and events have stimulated Key activities and events have stimulated interestinterest–DoD SE RevitalizationDoD SE Revitalization–AF Workshop on System RobustnessAF Workshop on System Robustness•Questions raised included:Questions raised included:–How do we show the value of Systems Engineering?How do we show the value of Systems Engineering? –How do you know if a program is doing good How do you know if a program is doing good systems engineering?systems engineering? •Sessions included SE Effectiveness measures and Sessions included SE Effectiveness measures and Criteria for Evaluating the Goodness of Systems Criteria for Evaluating the Goodness of Systems Engineering on a Program Engineering on a Program33Background of the Systems Engineering Leading Indicators Project““SE Leading Indicators Action Team” formed in late SE Leading Indicators Action Team” formed in late 2004 under Lean Aerospace Initiative (LAI) 2004 under Lean Aerospace Initiative (LAI) Consortium in support of Air Force SE Revitalization Consortium in support of Air Force SE Revitalization The team is comprised of engineering measurement experts The team is comprised of engineering measurement experts from industry, government and academia, involving a from industry, government and academia, involving a collaborative partnership with INCOSE, PSM, and several collaborative partnership with INCOSE, PSM, and several othersothers•Co-Leads: Garry Roedler, Lockheed Martin & Donna Rhodes, MIT Co-Leads: Garry Roedler, Lockheed Martin & Donna Rhodes, MIT ESD/LAI Research GroupESD/LAI Research Group•Leading SE and measurement experts from collaborative Leading SE and measurement experts from collaborative partners volunteered to serve on the team partners volunteered to serve on the team The team held periodic meetings and used the ISO/IEC 15939 The team held periodic meetings and used the ISO/IEC 15939 and PSM Information Model to define the indicators. and PSM Information Model to define the indicators. PSM (Practice Software and Systems Measurement) has PSM (Practice Software and Systems Measurement) has developed foundational work on measurements under developed foundational work on measurements under government funding; this effort uses the formats developed government funding; this effort uses the formats developed by PSM for documenting the leading indicatorsby PSM for documenting the leading indicators44A Collaborative Industry EffortA Collaborative Industry Effort… and several others55Objectives of the project1.1.Gain common understanding of the needs and drivers of this Gain common understanding of the needs and drivers of this initiativeinitiative2.2.Identify information needs underlying the application of SE Identify information needs underlying the application of SE effectiveness effectiveness –Address SE effectiveness and key systems attributes for systems, SoS, Address SE effectiveness and key systems attributes for systems, SoS, and complex enterprises, and complex enterprises, such as robustness, flexibility, and robustness, flexibility, and architectural integrityarchitectural integrity3.3.Identify set of leading indicators for SE effectiveness Identify set of leading indicators for SE effectiveness 4.4.Define and document measurable constructs for highest priority Define and document measurable constructs for highest priority indicators indicators –Includes base and derived measures needed to support each indicator, Includes base and derived measures needed to support each indicator, attributes, and interpretation guidanceattributes, and interpretation guidance5.5.Identify challenges for implementation of each indicator and Identify challenges for implementation of each indicator and recommendations for managing implementationrecommendations for managing implementation6.6.Establish recommendations for piloting and validating the new Establish recommendations for piloting and validating the new indicators before broad use indicators before broad use66SE Leading Indicator Definition•A measure for evaluating the effectiveness of a how a A measure for evaluating the effectiveness of a how a specific SE activity is applied on a program in a specific SE activity is applied on a program in a manner that provides information about impacts that manner that provides information about impacts that are likely to affect the system performance objectivesare likely to affect the system performance objectives–An individual measure or collection of measures that are An individual measure or collection of measures that are predictive of future system