What's in Your Test Tool?

Over the past few years the test automation tool market has become cluttered with tools that all claim to do the same thing: Automate Testing. Wikipedia lists 38 test framework tools for C/C++ alone. Unfortunately for potential users, when viewing product literature, or simplistic demos, many of these test tools look very much alike.

The purpose of this white paper is to provide data that engineers should consider when they are evaluating software test automation tools, specifically dynamic test automation tools.

You Can't Evaluate a Test Tool by Reading a Data Sheet

All data sheets look pretty much alike. The buzzwords are the same: "Industry Leader", "Unique Technology", "Automated Testing", and "Advanced Techniques". The screen shots are similar: "Bar Charts", "Flow Charts", "HTML reports" and "Status percentages". It is mind numbing.

What is Software Testing?

All of us who have done software testing realize that testing comes in many flavors. For simplicity, we will use three terms in this paper:

• System Testing: Testing the fully integrated application
• Integration Testing: Testing integration sub-systems
• Unit Testing: Testing a few individual files or classes

Everyone does some amount of system testing where they do some of the same things with it that the end users will do with it. Notice that we said "some" and not "all." One of the most common causes of applications being fielded with bugs is that unexpected, and therefore untested, combinations of inputs are encountered by the application when in the field.

To learn more, download the How To Evaluate Test Tools whitepaper.