Jeffrey Fortin

January 12, 2017

Create a Software Quality Scorecard for Your Organization

The balanced scorecard approach is used extensively in organizations worldwide as a performance measurement framework. As part of this approach, a number of Key Performance Indicators (KPI’s) are evaluated as a measurement to determine progress/success.

A simple KPI is reviewing stock price as an assessment of corporate value. If an interested party wants to know how any public company is doing, financial disclosures are readily available with Key Performance Indicators for the company published, such as stock performance over time, profitability, etc. In essence, this is a type of corporate scorecard. It’s easy for anyone to quickly assess the health of the company by looking at these KPIs.

In an increasingly software-driven world – where everything from mission- and safety-critical systems to our homes is being controlled by software-based products, wouldn’t it be great if we also had a Software Quality Scorecard? Just as a stock price is a measurable assessment of corporate value, bug count can serve as an equally recognizable assessment of code quality within the correct information parameters. By measuring quality and plotting it over time, a software development organization would have KPIs observable by outsiders (such as regulatory bodies and consumers). The organization itself would also be able to quickly assess where its stands in terms of its quality goals at any given moment.

Three Things You Can Do Today to Start Improving Your Software Quality

The easiest way to get started is to pick one or two KPIs and begin measuring them today. Pick something simple to start with that you can measure, plot and track over time.

Let’s take number of Static Analysis issues for example. Here are three things you can do today to start improving software quality:

  1. Start measuring your open Static Analysis issues count
    If you don’t measure quality, you can’t claim to have it. Don’t just say you have quality -- test for quality and show it. Plotting data over time can also help to predict the future. Therefore, once you begin measuring and testing for quality, start tracking it over time. Just as with stock charts, in software development, past performance can be used to predict the future – but an advantage over the stock market is that we have more control over the parameters of success.

  2. Publish your open Static Analysis issues count
    In this process, using tools such as an analytics dashboard to collect all of the quality and testing information (the results of unit testing, code coverage metrics, etc.), publishing this data and sharing with all internal teams is a powerful tool, particularly if you have the ability to do this in an easy and automated way. Furthermore, sharing this data with external audiences in the spirit of a “Software Quality Scorecard” as an observable performance metric will let the market know that your organization takes quality seriously.

  3. Set a goal to reduce your Static Analysis issues count backlog by 1% every month
    1% seems like a trivial goal but it beats no goal at all. Don’t be disappointed if it’s easy; keep this goal for at least three months. Once you succeed, set the next step -- maybe 2%. For even bigger gains, start to incorporate test automation to assess quality, and perform regression testing.

In closing, consider that every year, organizations commit themselves to key objectives. Oftentimes, this is achieved via metrics-based performance goals that incorporate leveraging best practices to streamline business process, which very well may include quality goals. Ultimately, measuring the impact these objectives have had on the organization involves some form of reporting – yet when it comes to developing software products, defining measurable goals and objectives for quality testing is often overlooked. Software isn’t going away; it is only growing in importance. Shouldn’t we place an equal level of importance on ensuring that it is high quality, and performs as expected – and have a measurable framework for evaluating that?