Software Quality

Jeffrey Fortin

February 09, 2017

It is widely known that testing is the best way to check the quality of your software, but how do you check the quality of your tests?

It may seem obvious, but software should always be thoroughly tested for quality. However, first-to-market advantage is a huge competitive factor and software developers are constantly under intense pressure to get products released quickly. As a result, testing is not always done properly in the rush to bring products to market. A large part of the issue is due to the misconception that investing more time in activities such as testing (which ensures increased quality) simply pushes out release dates by that same amount of time. This mindset neglects the value of testing over the lifecycle of the product. 

To help make testing a more efficient process for all, tests should be easy to understand, simple and quick to run, and preferably, run automatically. Here are some valuable ways you can test more efficiently while ensuring better quality.

Assessing Code Coverage
Code coverage is a highly effective way to measure the scope of your tests; after all, if you don't measure it, you can't know what it is.  Code coverage allows any portion of the application – or the entire application – to be analyzed at once, enabling the effectiveness of test efforts to be gauged by identifying which areas of an application were exercised during a test run. You may think you have completely tested your code if you pass all the tests, but facts may show otherwise. 

Code coverage has been proven to have one of the highest returns on investment of any development process change, and it can be a valuable addition to the software development process. Code coverage:

    • Provides insight into the completeness of testing
    • Provides reports to show whether released applications are fully tested
    • Forces developers to review code as tests are developed
    • Allows aggregation of code coverage from unit, integration, and system test

Modern tools make it easy to implement code coverage as part of the build process, allowing coverage measurement with no changes to the test process.

Mapping Tests to Requirements
Code coverage analysis goes hand-in-hand with requirements-based testing.  Have you tested all your requirements? Conversely, if you have tested all your requirements, what was the code coverage? 

Gaps in the code coverage might point to poorly implemented tests, which don’t adequately test the requirements. Maybe you are missing requirements, or have code that doesn’t need to be in the specific configuration, or maybe the code isn’t needed. For example, the gaps in code coverage might be the result of “extra” code (e.g., the requirement indicates there are 100 message IDs that should be handled, but the developer added 10 new ones and the requirement never got updated). 

Use Analytics to Make an Informed Decision to Improve Quality 
Analytics can help identify hot spots in code that have the greatest potential for error. Armed with the facts, you can add more tests, refactor the code to make it easier to understand, perform code reviews, run static analysis, and more. 

Sophisticated analytics tools enable organizations to drill down to analyze particular areas in the software that need to be addressed. This provides a “view of risk” such as an analysis of where the highest amount of complexity exists in the software. Naturally, the extremely complex areas equal the highest amount of risk. For example, if code coverage is being used to determine how well an application has been tested, analytics tools can determine that a particular function hasn’t been tested at all (0% code coverage), another one has only been tested up to 25% code coverage, and another one has been tested  with up to 50-75% code coverage. 

Having an easy to understand web-based dashboard of software code quality and testing completeness metrics can help to provide actionable intelligence for increasing quality. Furthermore, combining analysis with the ability to correct the worst problems and implement tests helps ensure that the problem does not recur.  

The simple fact is that you cannot have quality without complete testing; software quality and testing completeness are linked. Quantifiable software quality metrics and analytics can help companies make more informed decisions about where to allocate limited test resources, ultimately resulting in better quality software and faster releases.