2015 Software Testing Technology Report - Vector Software

Anna Barcelos

February 09, 2016

As the world around us increasingly becomes more dependent on the products whose functionality is controlled by software -- and the demand for improved product quality and regulations become more stringent than ever -- software testing is an industry in the midst of incredible transition.  

With all of the noise constantly going on in our industry, we wanted to take a step back and separate the hype from reality, and see what the embedded software testing industry really cares about right now, and what its concerns may be in the near future. As a result, Vector Software conducted a research survey designed to capture the thoughts of our industry leaders in relation to:

  • ƒWhat trends users are actually embracing in the real world
  • ƒWhat users are really interested in learning more about
  • ƒWhat are the projects that the market plans on addressing in the near future
  • ƒWhat actual capabilities are of interest to them now

The findings were pretty interesting. Our responses came from 285 industry professionals in the roles of management/project management, developers and testers/QA team. We looked at the data as a whole and by specific audience to glean some insights. The full report can be downloaded here, but the more notable findings are summarized below.

A topic that keeps a lot of our customers up at night is the issues associated with legacy code. Legacy code bases are a huge problem, and almost everyone in the industry knows that they are inadequately tested, but the problem often is they don’t know where to start in terms of fixing the problem. These legacy code bases have been accumulating some serious technical debt along the way, but some surprising results came in terms of respondents’ familiarity with the actual term “technical debt.”About 45% responded that they were “very unfamiliar” with the term (which represents latent defects introduced during system architecture, system design, or system development), and its overall weighted average score for familiarity came in at just 2.33 out of 5. 

However, when it came down to the actual capabilities that the audience was most interested in, 42% identified legacy code automated testing of interest in the survey overall, and the capability was also identified of interest by each individual audience as well, signaling the growing awareness for the need to test legacy code in an automated fashion. 

Additionally, 36% of the management audience stated that they were potentially planning to address “identifying project risk and technical debt” with upcoming projects, which likely indicates that the management level audience is starting to see, and feel, the financial impact of the accrued technical debt and will move quickly to establish a path forward by tackling the issues associated with those legacy code bases. 

The growth in the Internet of Things (IoT) has also had a noticeable effect on the software testing industry. When we asked participants to rate the terms that they were most familiar with,  the Internet of Things was the only non-specific test term that respondents rated with a high degree of familiarity (3.15 out of 5). Every other term they were very familiar with was specific to testing (Embedded Software Test; Continuous Integration; Continuous Testing; API Testing and Change Impact Analysis were the other top terms.)As the IoT enables the interconnection of the physical and virtual world, essentially every electronic device will have network connectivity – and that means that every manufacturer of electronic devices will also be in the software business. 

Also of interest were the top capabilities of interest identified by the market, which included regression and system testing; code coverage; automated C, C++, Ada dynamic/unit testing; and automatic test case generation. 

Our survey provides many more findings on a wide variety of key topics such as automatic test case generation, code coverage analysis, code quality and testing analytics, change-based testing, the importance of continuous integration and continuous testing and more, as well as a detailed look at the responses broken out between specific audience segments including management, developers, and testers/QA team.  We encourage you to download the complete report to learn more.