| ISSRE 2006 | START Conference Manager |
Knowing data-driven answers to these questions is vital to managing a software project from design through maintenance.
Borrowing from the principles of risk management in the insurance and financial worlds and a lot of hard work in analyzing previous releases of Windows by Microsoft Research, we took a manageable number of key metrics that have been proven to be good indicators of risk in Windows XP and Windows Server 2003, and created a toolset that regularly collects this information on each interim build of Windows Vista from eight different teams. The data this toolset produces allows us to better understand the changes occurring in the software. Change can be good or bad, so we determine the risk level of this churn by looking at several factors that characterize the churn, including amount and location, whether the code is fault prone (as measured by size, complexity and “usual suspect” metrics), its impact on partners and customers through operational profiling of how they use the changed code, bug fixes-to-feature churn payload ratios, and key quality metrics such as test results.
Scaling up to analyzing over 3500 binaries in Vista across eight teams, every day, continues to be a challenge, but we’re now able to generate a report within an hour of each release for each of the eight teams. This reports answers most of the questions posed with the data produced, focusing test teams, coordinating integration and establishing a risk level for each build of Vista.
| START Conference Manager (V2.52.6) |
| Maintainer: mark.sherriff@ncsu.edu |