Automating Software Testing with VectorCAST
The VectorCAST embedded software testing platform is a family of products that automates testing activities across the software development lifecycle. The purpose of this section is to provide a high-level view of what the VectorCAST products do, how they interwork, and how you might best use them to fully meet your own software testing needs.
Advantages
- Focus on Embedded Systems
- Enable Continuous and Collaborative Testing
- Provide the most Automated Tools possible
- Works with your existing software development tools
Application Areas
Structural Coverage Certification for Airborne Systems and Equipment
What is RTCA DO-178C; EUROCAE ED-12C
One of the key requirements in the software verification process of DO-178C and ED-12C is achieving structural code coverage in conjunction with the testing of the highlevel and low-level software requirements.
These documents classify software into five levels of criticality based on whether atypical software behavior could cause or contribute to the failure of a system function. The table below shows the relationship between the failure condition category and the structural coverage objective as defined the standards.
Software Criticality Level and Structural Coverage Requirements
Level | Failure Definition | Associate Structural Coverage |
---|---|---|
A | Software resulting in a catastrophic failure condition for the system | Modified Condition / Decision Coverage, Decision Coverage and Statement Coverage |
B | Software resulting in hazardous or severe-major failure condition for the system | Decision Coverage and Statement Coverage |
C | Software resulting in a major failure condition for the system | Statement Coverage |
D | Software resulting in a minor failure condition for the system | None Required |
What is VectorCAST for DO-178C and ED-12C
The VectorCAST family of tools supports the creation and management of test cases to prove that the low-level software requirements have been tested. Additionally, the VectorCAST tools support the capture and reporting of structural code coverage data at all levels prescribed by DO-178C and ED-12C including Level A.
Highlights
- Supports all levels of code coverage required by DO-178C; ED-12C
- Automatic generation and compilation of complete test stubs and driver programs
- Automatic regression testing
- Compliant test report generation
- Test execution on host, simulator, and embedded target system
VectorCAST Tool Qualification Deliverable Process
DO-178C and ED-12C specifies that qualification of a software tool is necessary when processes prescribed by the document, are eliminated, reduced, or automated by the use of a tool, without its output being manually verified. Because VectorCAST automates the process of validating low-level requirements and the process of recording and reporting on structural code coverage, it must be qualified when used as part of a DO-178C and ED-12C certification. Vector Software has developed an off-the-shelf qualification package that has been used on more than forty DO-178C and ED-12C certification projects.
Download Using VectorCAST for DO-178B/C Software Verification White Paper
Software Verification and Validation for Automotive Functional Safety
What is ISO 26262
The International Organization for Standardization document 26262 (ISO 26262) provides a process framework and a procedural model for the development and test of safety critical electronic systems in motor vehicles. It is a derivative of the International Electronic Commission document 61508 (IEC 61508) adapted for the challenges met in Automotive. VectorCAST for ISO 26262 satisfies the recommendations for software testing and verification specified in Part 6 of the standard for software development. Recommended activities include both unit level and system level testing activities such as functional tests (requirement-based tests and partition tests) and structural coverage tests.
Highlights
- Supports all levels of code coverage required by ISO 26262
- Statement
- Branch
- MCDC
- Automatic generation and compilation of complete test stubs and driver programs for unit and integration testing
- Automatic regression testing
- Requirement to test case mapping
- Robustness Testing
- Out of bounds values
- Boundary condition testing
- Test execution on host, simulator, and embedded target systems
What is VectorCAST for ISO 26262
The VectorCAST family of tools supports the creation and management of test cases to prove that the low level software requirements have been tested and is also useful for a variety of robustness testing activities such as range and out of bounds testing. Additionally, the VectorCAST tools support the capture and reporting of structural code coverage at all levels required by ISO 26262.
How VectorCAST Supports ISO 26262
VectorCAST accelerates testing efforts related to ISO 26262 compliance by empowering users with the following benefits:
- Complete support for all levels of structural coverage
- Automatic generation and compilation of test stubs and driver programs for functional and robustness testing
- Requirements can be tagged to test cases
- Automatic generation of type value range test cases (min-mid-max)
- Test cases based on plausible range of values
- Interactive point-and-click and script generated test case construction
- Test case modification and re-execution without re-compilation
- Automatic regression testing
- Standards compliant test report generation
- Basis path analysis and cyclomatic complexity
- Test execution on both host and embedded target development systems
TÜV SÜD Certified Tool for ISO 26262 Safety Related Development
The VectorCAST product family is a TÜV SÜD certified software tool for safety related development.
Download Following the ISO 26262-6:2018 Guidelines with VectorCAST White Paper
Functional Safety Certification for Programmable Electronic Systems
What is IEC 61508
IEC 61508 is the international industry standard governing functional safety of programmable electronic systems. IEC 61508 certification confirms that a product or system complies with objectives set by the standard. IEC 61508 is comprised of seven (7) parts. In “Part 3: Software requirements” (IEC 61508-3), the document defines software requirements and sets the safety lifecycle for software, including validation and verification. The safety lifecycle begins with a risk analysis to determine the Safety Integrity Level (SIL) required. IEC 61508-3 highly recommends certified tools and translators for safety integrity levels of SIL-2 and higher.
Highlights
- Supports all levels of code coverage required by IEC 61508
- Statement
- Branch
- MCDC
- Automatic generation and compilation of complete test stubs and driver programs for unit and integration testing
- Automatic regression testing
- Requirement to test case mapping
- Robustness Testing
- Out of bounds values
- Boundary condition testing
- Test execution on host, simulator, and embedded target systems
What is VectorCAST for IEC 61508
The VectorCAST embedded software test platform supports the creation and management of test cases to prove that the low level software requirements have been tested and is also useful for a variety of robustness testing activities such as range and out of bounds testing. Additionally, the VectorCAST tools support the capture and reporting of structural code coverage at all levels required by IEC 61508.
Level | Consequences Failure |
---|---|
0 | N/A |
1 | Potential for minor injuries |
2 | Potential for major injuries or one fatality |
3 | Potential for multiple fatalities |
4 | Potential fatalities in the community |
Four levels of SIL to specify the necessary safety measures for avoiding an unreasonable residual risk
How VectorCAST Supports IEC 61508
VectorCAST accelerates testing efforts related to IEC 61508 compliance by empowering users with the following benefits:
- Complete support for all levels of structural coverage
- Automatic generation and compilation of test stubs and driver programs for functional and robustness testing
- Requirements can be tagged to test cases
- Automatic generation of type value range test cases (min-mid-max)
- Test cases based on plausible range of values
- Interactive point-and-click and script generated test case construction
- Test case modification and re-execution without re-compilation
- Automatic regression testing
- Standards compliant test report generation
- Basis path analysis and cyclomatic complexity
- Test execution on both host and embedded target development systems
TÜV SÜD Certified Tool for IEC 61508 Safety Related Development
The VectorCAST product family is a TÜV SÜD certified software tool for safety related development.
Download Understanding Verification and Validation of Software Under IEC 61508-3:2010 White Paper
Software Safety Certification for Medical Devices and Equipment
A wide variety of medical devices incorporate embedded software. Some of these such as: X-ray scanners, heart rate, blood pressure and breathing monitors, are used for analyzing patients. Others such as defibrillator and medication pumps, are used to treat patients.
In both cases, people’s health and safety depend on the medical software deployed being reliable, safe and bug-free.
Medical devices sold in the United States must be approved by the U.S. Food and Drug Administration (FDA). The FDA has a formal process for device manufactures to follow for certification including guidelines for software validation.
What is IEC 62304
The International Electrotechnical Commission created the IEC 62304 standard. The standard, formulated to govern the requirements for medical software, describes the process that medical software must go through in order to be approved for use in Europe. The standard includes guidelines for testing any software components that are part of a medical device.
Highlights
- Supports all levels of code coverage required by IEC 62304
- Statement
- Branch
- MCDC
- Automatic generation and compilation of complete test stubs and driver programs for unit and integration testing
- Automatic regression testing
- Requirement to test case mapping
- Robustness Testing
- Out of bounds values
- Boundary condition testing
- Test execution on host, simulator, and embedded target systems
What is VectorCAST for FDA and IEC 62304
The VectorCAST embedded software test platform supports the creation and management of test cases to prove that the low level software requirements have been tested and is also useful for a variety of robustness testing activities such as range and out of bounds testing. Additionally, the VectorCAST tools support the capture and reporting of structural code coverage at all levels recommended by IEC 62304.
Proving Due Diligence for FDA Audits
VectorCAST tools are successfully used by clients demonstrating due diligence with FDA software quality requirements. Whether your device is Class I, II or III, the VectorCAST tools provide a dependable and repeatable testing process for your medical device software development.
TÜV SÜD Certified Tool for IEC 62304 Safety Related Development
The VectorCAST product family is a TÜV SÜD certified software tool for safety related development.
Download Using VectorCAST to Satisfy FDA Software Testing Requirements White Paper
Download the TÜV SÜD Certification Safety Manual
Download Using Vector Tools for Medical Software Certification According to IEC/EN 62304 White Paper
Software Verification and Validation for Railway Applications
What is EN 50128:2011 and EN 50657:2017
The European Standards EN 50128:2011 and EN 50657:2017 provide a process framework and a procedural model for the development and test of safety critical electronic systems in railway systems. It is a derivative of the International Electronic Commission document, IEC 61508 adapted for the challenges met in railway systems.
Highlights
- Supports all levels of code coverage required by EN 50128 and EN 50657
- Statement
- Branch
- MCDC
- Automatic generation and compilation of complete test stubs and driver programs for unit and integration testing
- Automatic regression testing
- Requirement to test case mapping
- Robustness Testing
- Out of bounds values
- Boundary condition testing
- Test execution on host, simulator, and embedded target systems
What is VectorCAST for EN 50128 and EN 50657
The VectorCAST embedded software test platform supports the creation and management of test cases to prove that the low level software requirements have been tested and is also useful for a variety of robustness testing activities such as range and out of bounds testing. Additionally, the VectorCAST tools support the capture and reporting of structural code coverage at all levels required by EN 50128 and EN 50657.
How VectorCAST Supports EN 50128 and EN 50657
VectorCAST accelerates testing efforts related to EN 51208 and EN 50657 compliance by empowering users with the following benefits:
- Complete support for all levels of structural coverage
- Automatic generation and compilation of test stubs and driver programs for functional and robustness testing
- Requirements can be tagged to test cases
- Automatic generation of type value range test cases (min-mid-max)
- Test cases based on plausible range of values
- Interactive point-and-click and script generated test case construction
- Test case modification and re-execution without recompilation
- Automatic regression testing
- Standards compliant test report generation
- Basis path analysis and cyclomatic complexity
- Test execution on both host and embedded target development systems
TÜV SÜD Certified Tool for EN 50128 and EN 50657 Safety Related Development
The VectorCAST product family is a TÜV SÜD certified software tool for safety related development.
Highlights of VectorCAST 2024
Coded Tests
- In the 2024 release of VectorCAST/C++, a significant enhancement is the introduction of Coded Tests, designed to complement the existing VectorCAST Test Editor. This feature supports the creation of test cases directly in code, leveraging the xUnit framework for streamlined setup and teardown in C++ unit testing. It also enables the efficient sharing of objects between tests, improving test organization and efficiency within the context of unit testing workflows.
Test Case Variants
- VectorCAST 2024 introduces a flexible approach to executing subsets of unit tests tailored to specific environment configurations. Utilizing a Logics File, this feature allows for precise control over which test cases are executed in a given unit test environment. Each Test Case Variant represents a unique combination of the unit test environment and the test cases applicable to it, as determined by the environment's Logics File. This functionality enhances the specificity and efficiency of unit testing, providing a refined mechanism to manage and execute tests that are most relevant to the current testing conditions.
Improve Unit Testing Efficiency with VectorCAST 2024
Try VectorCAST Webinar Recording: 'What's New in VectorCAST 2024Product Descriptions
Platforms
VectorCAST and the FlexNet Publisher distributed with VectorCAST is available for two platforms, 64-bit Windows and 64-bit Linux.
Windows 64-bit
The following Windows platforms are supported (64-bit version only):
Windows 11
Windows Server 2019
Windows 10
Windows Server 2012 R2
Windows 8.1
Windows Server 2012
Windows 8
Windows Server 2008 R2 Service Pack 1 (SP1)
Windows 7 SP1
Windows Server 2008 Service Pack 2 (SP2)
Windows Vista SP2
For the Windows 64-bit VectorCAST, your machine will need to have the Windows Universal C Runtime (CRT) installed. Universal CRT Deployment | Microsoft Docs
Note that if you are using FlexNet Publisher 11.16.4, then the Windows Universal C Runtime (CRT) is required.
Linux 64-bit
For Linux host platforms, the minimum system requirements are:
A 64-bit distribution with the GNU C Library (glibc) version 2.5 or later.
FlexNet Publisher requires that the LSB package for your Linux platform be installed.
Certain parts of VectorCAST---mainly those related to GUIs driven by Python---rely on the Qt5 toolkit, which has further system-wide dependencies. VectorCAST functionality relying on Qt5 includes features such as the RSP Configurator and Requirements Gateway integrations. To run these features, VectorCAST follows the minimum requirements for Qt5:
GNU C Library (glibc) version 2.17 or later.
Kernel version of 3.10.0 or later.
These dependencies are not mandatory and are only necessary if you wish to run Qt5-driven functionality.
Depending on your Linux distribution, the following packages are likely to be required:
For SUSE-based distributions (e.g., SLES or OpenSUSE):
libxcb-image0
libxcb-keysyms1
libxcb-randr0
libxcb-render0
libxcb-render-util0
libxcb-shape0
libxcb-shm0
libxcb1
libxcb-sync1
libxcb-util1
libxcb-xfixes0
libxcb-xinerama0
libxcb-xinput0
libxcb-xkb1
For RedHat-based distributions (e.g., RHEL, CentOS, Oracle Linux):
libxcb
xcb-util
xcb-util-image
xcb-util-keysyms
xcb-util-renderutil
For Ubuntu/Debian-based distributions:
libxcb1
libxcb-image0
libxcb-keysyms1
libxcb-randr0
libxcb-render0
libxcb-render0-util0
libxcb-shape0
libxcb-shm0
libxcb-sync1
libxcb-util1
libxcb-xfixes0
libxcb-xinerama0
libxcb-xinput0
libxcb-xkb1
Do you have technical questions? Visit our KnowledgeBase to find the answers.
VectorCAST/QA
System Test Automation/Code Coverage
VectorCAST/QA enables teams to implement consistent and repeatable processes for managing test activities and reporting key quality metrics such as code coverage. Test automation simplifies test execution. It is compatible with all test execution frameworks. VectorCAST/QA also provides an integrated workflow for white-box system testing. Application internals can be monitored, and faults injected using Probe Points.
TESTinsights
Collaborative Analysis of Test Result Data
TESTinsights extends the functionality of VectorCAST to accumulate all results of software examination and software test activities from all participants with a collaborative service. Participants analyze these result data collaboratively in web dashboards to gain insights from different points of views and levels of abstraction.
VectorCAST/Coupling
Data and Control Coupling Verification
VectorCAST/Coupling provides automated analysis and instrumentation for data coupling and control coupling verification for C and C++ source files and is used in conjunction with an existing VectorCAST/QA project or VectorCAST Cover environment. Coupling verification is mandatory for safety-critical avionics software development according to both RTCA DO-178B and DO-178C. The tools support the Coupling requirements of DO-178B/C by using a combination of static analysis to identify the couples in a code base and run-time verification of the couples during application execution.
VectorCAST Test Explorer
Run VectorCAST/C++ Test Cases From Visual Studio Code
Would you like to run VectorCAST tests using the VS Code Test Explorer UI? You can do this by adding the VectorCAST Test Explorer Extension.
VectorCAST Test Explorer is an open-source extension for Visual Studio Code that allows you to create, edit, execute, and debug VectorCAST/C++ test cases from within VS Code.
Success Story
Komatsu Implements VectorCAST for the Unit Testing of Next Product Development
Also Uses it in the Time-series Testing to Validate a Series of Behaviors
Komatsu Ltd. has implemented Vector's VectorCAST as a unit testing tool, following the change of its development language for construction machinery controller software from C language to C++. It has also conducted time-series testing to validate a series of behaviors such as filtering by using the VectorCAST compound testing function.
Downloads
Please see the VectorCAST Downloads Page on our website to access the VectorCAST Product Installers.
News & Events
Compilers
Compilers, Hardware, Debuggers and Simulators
VectorCAST supports C, C++ and Ada compilers. VectorCAST is user-configurable to match your development toolchain including hardware and simulators.
Qualification Package
Tool Qualification and Verification packages for aerospace, railway, industrial, automotive, and medical.
The VectorCAST Tool Qualification package for DO-178B and DO-178C contains tool qualification documentation, test cases, and procedures that let you qualify VectorCAST tools on projects based on the DO-178B/ED-12B or DO-178C/ED-12C standard. The package contains the Tool Operational Requirements (TOR) document and Tool Qualification Document (TQD) along with other documents required by DO-330. With the package, development organizations can streamline certification of their embedded systems developed using VectorCAST.
The VectorCAST Tool Validation Package for IEC 61508, ISO 26262, EN 50128, FDA and IEC 62304 provides documentation, certificates, and templates that help you use VectorCAST on projects based on the IEC 61508, ISO 26262, EN 50128, FDA and IEC 62304 standards and FDA guidelines. The package includes certificates and reports from the certification authority TÜV SÜD that are based on documented, application-specific verification workflows. The VectorCAST Tool Validation Package offers organizations the required evidence to demonstrate compliance with the applicable standards.
Tool Integrations
MathWorks Simulink®
Simulink is a block diagram environment for multi-domain simulation and Model-Based Design. It supports system-level design, simulation, automatic code generation, and continuous test and verification of embedded systems. Simulink provides a graphical editor, customizable block libraries, and solvers for modeling and simulating dynamic systems.
Ansys SCADE®
Ansys SCADE® - Ansys SCADE Suite is a model-based development environment for reliable embedded software, which provides linkage to requirements management, model-based design, verification, qualifiable/certified code generation capabilities and interoperability with other development tools and platforms.
Visure Requirements
Visure Requirements ALM integration with VectorCAST - Empower engineering teams building complex products or systems to sync testing processes without losing end-to-end traceability by integrating Visure Requirements ALM with VectorCAST. Create consistency, alignment and empower teams throughout the development and testing process by making cross-functional data available to users of both systems in real-time, resulting in a more efficient effective, and successful product outcome.
Polarion® REQUIREMENTS™
Polarion REQUIREMENTS is a unified collaborative solution encompassing both Requirements Management and Test Management in a single web-based platform. Forensic level artifact linking and fully automated history guarantee traceability, accountability, transparency and easier compliance with regulations and standards.
IBM® DOORS®
IBM® DOORS is a requirements management software optimized for the needs of complex and embedded systems development. IBM® DOORS enables you to capture, trace, analyze and mange changes to requirements and helps you demonstrate compliance to regulations and standards.
DOORS Next
IBM Engineering Requirements Management DOORS Next (DOORS Next) is a requirements management tool that helps you store, categorize, link and share product requirements with stakeholders such as reviewers, designers, testers, and developers.
IBM® Rational® RequisitePro®
IBM Rational RequisitePro is a requirements management tool. Rational RequisitePro helps project teams to manage their requirements, to write good use cases, to improve traceability, to strengthen collaboration, to reduce project rework, and to increase quality.
PTC®
The customizable traceability of PTC Integrity provides a single source of truth that keeps all stakeholders informed of changes to requirements throughout the development process, fostering collaboration between engineering disciplines and roles. Developers and other stakeholders can easily assess and address the impact of changes before they are implemented, even across product lines, to increase agility, improve product quality and reduce product delivery risk.
Jama Software®
Jama Software offers a robust solution for requirements management and end-to-end product delivery. With Jama tools as your central hub for collaborative requirements management, your product teams can capture, discuss and manage all your requirements in one place and connect everything together with end-to-end traceability and impact analysis.
Intland Software codeBeamer
codeBeamer ALM is a collaborative requirements management & ALM tool that offers robust features throughout the entire development lifecycle. Specify, manage, and trace the coverage of requirements throughout the process of development. codeBeamer offers gapless end-to-end traceability, collaboration, code and test coverage whether you're using Waterfall, Agile, or a scaled Agile method. Easy to integrate into any tool environment, codeBeamer is a disruption-free, fast, and scalable platform of collaboration to align and streamline your development processes.
Jenkins®
Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:
- Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
- Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.
Wind River® Simics®
Simics is a full system simulator helping companies increase their productivity by transforming software development processes to be faster and more efficient. When used in combination with VectorCAST, developers can create a Massively Parallel testing infrastructure that ensures testing is completed in minutes rather than days. VectorCAST and Simics enable tests to be easily run by every developer on every source change, resulting in a seamless solution for deploying high quality hardware and software applications.
The VectorCAST integration with Simics sits on top of a Continuous Integration server, such as Jenkins, controlling the test dispatch to a pool of Simics targets. By adopting this integrated approach using Simics, teams can overcome a variety of testing challenges arising from increased test complexities or limited hardware. This provides a single point of control for testing, allowing engineers to quickly and easily test an entire device. The integrated toolchain is of particular value to engineers working with complex or safety-critical applications with long life-cycles.
Bamboo®
Delivering JIRA issues – from code to customers, Bamboo does more than just run builds and tests. It connects issues, commits, test results, and deploys so the whole picture is available to your entire product team – from project managers, to devs and testers, and sys admins.
Static Analysis Tools
Vector offers PC-lint Plus, an industry-known, trusted name in Static Analysis.
VectorCAST also integrates with a number of other static analysis tools, in addition to PC-lint Plus, to perform module-based or whole-program source code analysis on C and C++ codebases and automatically identifies problems at their source, prior to compilation.
Supported RTOS
VectorCAST is designed to support any commercial-quality real-time operating system (RTOS).
The VectorCAST Runtime Support Package VectorCAST RSP, provides a full-featured integration that allows for the download, execution and result capture using the built-in networking facilities of the target RTOS.
The table below is a sample of the RTOSes supported by VectorCAST. These RTOSes are provided by a number of vendors and are available under various commercial or open source terms. Please visit the providers’ website for more information.
INTEGRITY® | MQX™ | EUROS RTOS™ | ThreadX™ |
FreeRTOS™ | LynxOS™ | µC/OS™ | Windows® XP Embedded |
Hard Hat Linux™ | Neutrino® | RTEMS™ | embOS™ |
ELinOS | DSP/Bios™ | SafeRTOS® | VxWorks® 5.4, 6.x |
NEOS™ | Embedded Linux | Windows CE™ |
|
PikeOS | DEOS | RTX |
Training
VectorCAST Workshops
Our workshops offer you many opportunities to expand or deepen your knowledge about VectorCAST.
Gain a basic understanding of Unit Testing and Code Coverage concepts and how the VectorCAST test automation platform can be used to implement those concepts.