The Effect of Rapid Commercial Off-the-Shelf (COTS) Software Insertion on the Software Reliability of Large Scale Undersea Combat Systems
Kevin Mattos, Christine Moreira, Mark Zingarelli and Denis Coffey
The 17th IEEE International Symposium on Software Reliability Engineering (ISSRE 2006) -- Government Track
Raleigh, North Carolina, USA, 7-10 November 2006
Abstract
Recent paradigm shifts in Department of Defense and Navy acquisition of large-scale critical undersea software systems have called for the insertion to the maximum extent possible of the latest Commercial Off-the-Shelf (COTS) software products. This new philosophy redirects primary focus from highly reliable, but labor intensive and expensive, software systems to increasingly more functional and cost effective systems. The result is a configuration management plan calling for a new software drop every year using the latest version of pertinent commercial software products. The heavy use of non-open source products creates a software reliability dependency on the commercial products being used. Thus, overall system reliability numbers cannot achieve a reliability threshold greater than the combination of all COTS products utilized. This also creates a heavy dependency on patches and fixes delivered by commercial vendors to increase software reliability. The release of these fixes is more often driven by the commercial market as opposed to being called for by the Navy itself.
This paper examines the effects of the rapid COTS insertion process on the overall system software reliability in undersea combat systems and presents a reliability growth model of the systems developed under this process. The contrast between the reliability of previous non-rapid COTS insertion paradigm systems and the current COTS based systems is discussed. A trade off of functionality versus reliability results in insufficient time allowed for the software system to mature. Specific cases are presented in which software reliability growth is observed across a set of testing periods using minimal software version change. Those cases are compared against the effect of issuing a major software drop with COTS version changes shortly following the initial testing periods in which a software reliability decrease was observed. The effect of such a paradigm shift on software reliability requirements thresholds is also discussed.