Continuous Integration and Test

Continuous Testing improves software quality and reduces time-to- market, by ensuring that every code change is completely tested prior to being committed to configuration control.

For years, software development has been a linear process with three distinct phases: design, code, and test. The time lag between coding and testing, means that hundreds of bugs can accumulate between test cycles, and these faults are only found weeks later during the next test cycle. This often results in the test phase lasting longer than the coding phase.

The VectorCAST Jenkins integration enables development groups to implement a continuous test process, where each code change is completely tested against all existing test cases prior to integration. This results in the shortest possible feedback loop for the developer, allowing bugs to be identified within minutes of code changes, rather than weeks.


What is Continuous Testing?

Most software development groups have implemented a Continuous Integration (CI) process to reduce total build times.  A Continuous Integration workflow provides incremental and parallel source code compilation resulting in greatly reduced application build times.  Continuous Integration tools allow developers to only compile the changed files, and those compilation jobs are spread over multiple cores on a single server, or multiple physical servers.  Continuous Testing applies these same techniques to software testing, dramatically reducing test cycle times by intelligently selecting the sub-set of tests that are affected by a set of code changes, and then running those tests in parallel.

Using a Continuous Integration Server

A CI server acts as a job broker to distribute individual build or test jobs to a farm of machines.  A widely used CI server is the open-source Jenkins tool, which is built with Java, and provides over 400 plugins to support building and testing of virtually any project. All CI servers provide the following features:

  • The ability to define a list of “nodes” which are physical or virtual machines
  • The ability to “tag” a node to indicate the kind of jobs it can run
  • The ability to remotely dispatch and balance jobs to a list of nodes
  • The ability to e-mail job status when jobs are complete

Using VectorCAST with a Continuous Integration Server

The VectorCAST Continuous Testing features leverage whatever CI Server your project uses to allow any sub-set of these tests to be run using any available configuration on any available target in parallel.  VectorCAST provides the following Continuous Testing infrastructure features:

  • The ability to organize thousands of test cases into logical groups that conform to the structure of your application
  • The ability to determine the pass/fail status of each test case
  • The ability to determine code coverage on a per function basis
  • The ability to run the same test against multiple configurations of the application
  • Command Line and Python API interfaces to all of these features

The Benefit of Continuous Testing

Continuous Testing improves software quality and time-to-market, by finding bugs immediately when they are introduced rather than weeks later during the next "test cycle"