ISSRE 2006 START Conference Manager    

Characterizing Software Defect Repair Times

Robert Mullen

The 17th IEEE International Symposium on Software Reliability Engineering (ISSRE 2006) -- Industry Practices (ISSRE 2006)
Raleigh, North Carolina, USA, 6-10 November 2006

Abstract

Our problem was to characterize and improve software defect repair times order to improve both reliability of released networking products and time-to-market of products under development. We measure repair times two ways at Cisco. The first, Mean Time to Repair, applies Little’s law, dividing the average number outstanding by the close rate to create a metric analogous to business metrics like Days-Account-Receivable. Alternatively we study the distribution of the Ages of defects at the time they were fixed. We point out the advantages of both methods but will concentrate on the use and properties of Ages here.

The distribution of Ages exposes the degree of inconsistency in responsiveness to defect reports. Typically the Age distribution is skewed, having a mean much larger than the median. The lognormal is often proposed as a repair-time distribution, but rarely is there any empirical evidence. We found the lognormal an excellent fit and provide the evidence.

A lognormal distribution of the Ages is characterized by the mean of the log Age and the variance of the log Age. Although they are independent, the actual (i.e. non log) values of both the mean Age and the variance are strongly affected by the variance of the log Age. Lesson: the best way to control the mean is to control the variance of the log Age, and vice versa. The lognormal arises when a value, such as Age, is dependent on a variety of multiplicative factors, for example: clarity of bug description, severity, complexity of fix, programmer speed, domain expertise, and availability of resources. Lesson: control the variance of the inputs to control the results --- this will be harder. Finally we provide enough examples to uncover situations when this model breaks down as well as noting its implications for reliability.

  
START Conference Manager (V2.52.6)
Maintainer: mark.sherriff@ncsu.edu