COTS Integration:How Far Have We Come?Problems and Solutions in 2002GSAW 2002 COTS IntegrationBreakout Session SummarySuellen EslingerKaren OwensThe Aerospace CorporationContact: [email protected] or [email protected]© 2002, The Aerospace Corporation2Breakout Session Goals•View the topic of COTS integration from multipleperspectives❖Integrator❖Vendor❖Cost❖Participants•Identify solutions and successes❖What factors contributed to the successes?•Identify ongoing and new problems and issues❖What factors contributed to the problems?❖How can the problems be reduced or eliminated in the future?•Update survey on COTS upgrade releasefrequency3What Happened•Introductions❖Session participants included Aerospace, industry, academia,acquirers, users, and cost estimators•Integrator perspective❖"Integration and Maintenance with the COBRA Architecture” -Tim Spinney, The Aerospace Corporation•Vendor perspective❖"Integration: Reinventing an Expensive Wheel”Larry McQueary, Talarian•Cost perspective❖"Costing COTS Integration" - Marilee Wheaton, TRW•32 participants with lively discussion•Collected data for COTS upgrade release survey4“Integration and Maintenance with theCOBRA Architecture” -- Tim Spinney•Maintenance paradigm not updated to reflect newrealities of COTS-Based System (CBS)sustainment❖Insufficient funds budgeted for COTS upgrades❖Management reassigns COTS upgrade budgets to effortsconsidered higher priority❖License costs are a large portion of minimal maintenancebudget•Solutions:❖Middleware-based architecture with loose coupling offunctional areas❖Small number of COTS products❖Written back-out plan for every COTS product upgrade5“Integration: Reinventing an ExpensiveWheel” -- Larry McQueary•Solutions:❖Don’t overlook or underestimate the infrastructure of thesoftware architecture❖Use loosely coupled components❖Adhere to standardized interfaces❖Thoroughly evaluate COTS products (including possiblevendor relationships)❖Prototype use of COTS products❖Enter into a partnership with the vendor and communicate❖Utilize vendor training and consulting capabilities✦Don’t develop in a vacuum❖Buy the maintenance that you need (e.g., 24/7 support)“Despite discussion to the contrary, using COTS is stillcheaper and more effective than building”6“Costing COTS Integration” -- Marilee Wheaton•Use lessons already learned•Use cost risk analysis to estimate cost of uncertaintiesin CBS development❖Define, estimate and analyze risk in each component•Use cost models that incorporate COTS-related costs(e.g., COCOTS, SEER-SEM)•Need to estimate size of COTS (not just SLOC), amountof functionality used, and COTS cost drivers•Collect COTS integration data to provide a historicalbasis for better estimates❖Calibrate the cost models with your data7“Costing COTS Integration” -- Marilee Wheaton•Be sure to include all COTS-related costs,especially those not included in the cost modelestimates (e.g., licenses, training)•Expect glue code to have a lower productivity thancustom software•Cost of COTS versus custom development needs tobe evaluated for the full life cycle (not justdevelopment)❖Distribution of costs will be different for COTS and customdevelopment projects8Conclusion•CBS development and sustainment has inherentuncertainties beyond the control of the acquirers,developers and users•Effective CBS development and sustainmentrequires a change of processes and attitudesacross the entire life cycle and among all parties❖Acquirers❖Developers❖Users❖Integrators❖Procurement❖Contracts9Survey on COTS Upgrade Release Frequency1. In your experience, what is the average durationbetween releases of a given COTS Product?❖GSAW 99 = 6.3 months❖GSAW 00 = 8.5 months❖GSAW 01 = 8.75 months❖GSAW 02 = 9.6 months (range: 6 to 18)2. For system(s) with which you have experience,how frequently are system upgrades releasedthat incorporate COTS software upgrades?❖GSAW 02 = 20.5 months (range: 4 to 70)10Backup ChartsProductMarketplaceResources ProcessIntergroupInteractionKey for following charts:•“Gn” indicates GSAW 2001 issue and ranking•“A” indicates 2000 Aerospace study issue12Product Issues•Requirements vs. COTS capabilities (G1)•Integration of❖Multiple COTS products (G2)✦Incompatibilities among COTS products (A)❖COTS products with new/reuse software (G5)•COTS independent architecture (G3)❖Designing architectures for COTS evolution (A)❖Designing in safety, security, supportability (A)•Mission risk (G6)•Cross platform portability (G11)•Standards: good, bad, ugly? (G14)•API breakage (“unplug and replay”) (G15)•Dormant functionality or features (G18)•Adverse effects of product upgrades on system (A)13Process Issues - Developer•Robust initial and periodic COTS product evaluation (A)•Prototyping in a system context (A)•Testing in operational context (G12)❖Regression testing of upgrades in system context (A)•Adapting software and systems engineering processesfor CBS development and maintenance (A)•Still need systems and software engineering (A)•Need enhanced CM processes (A)•Planning for COTS upgrades and evolution duringdevelopment and maintenance (A)•Selection of hardware platforms with availability ofCOTS software as key criterion (A)14Process Issues - Customer and User•Acquisition and support strategies (G16)•Adapting customer/user processes to CBSacquisition, operations and maintenance (A)•Processes for trading cost, schedule, requirements,and O&M concepts (A) against COTS capabilities❖Need requirements prioritization (A)•Need contracts compatible with CBS development andmaintenance (A)•Standardized processes for safety certification andsecurity accreditation of CBS needed (A)•Standardized license processes to ensure suitability oflicenses and maintaining currency (A)15Resource Issues•Cost vs. benefit of upgrading (G7)•Acquiring and maintaining CBS skills (G9)•Accurately costing all aspects of CBS developmentand maintenance (A)•Optimal scheduling of upgrades (A)•Increased computer resources for upgrades (A)•Modifying COTS is a BAD idea! (A)•Need cost and schedule management reserves (A)•Reallocating time and effort across life cycle (A)❖More time for evaluation, prototyping and analysis (A)❖Less time for implementation; more time for integration (A)16Marketplace