We evaluated a number of test tools and were impressed with the VectorCAST automated solution. Using VectorCAST/C++, we have significantly...Read More »
Ensure Your Medical Device Embedded Software Has Healthy Compliance with Software Quality Requirements
Modern medicine increasingly relies on electronic devices to improve quality of life and medical care. Your devices provide life saving capabilities: patient monitors, infusion pumps, pacemakers, and defibrillators. The common element is all contain significant amounts of embedded software.
These devices need to behave properly and consistently, otherwise the well-being of patients is compromised. Testing is a critical part of ensuring the performance of these devices to pass FDA audits and support the guidelines defined in IEC 62304.
The Complete Solution for Software Testing of Embedded Medical Applications
VectorCAST/C++ allows you to perform unit testing of individual functions or groups of functions to ensure that your application is meeting its requirements, without error. VectorCAST/C++ provides both pass/fail analysis as well as code coverage analysis to support the testing of any class of medical device.
VectorCAST/Cover enables your engineers to assess the thoroughness of system testing by automatically recording which parts of the application are stimulated by each system test. VectorCAST/Cover supports statement, branch and Modified Condition/Decision Coverage (MC/DC) coverage levels. VectorCAST/MCDC coverage was originally developed for the Avionics industry and now many leading medical devices companies have adopted this as a code coverage standard for their FDA Class III or IEC 62304 Class C applications.
Proving Due Diligence for FDA Audits
VectorCAST automated testing tools are successfully used by clients demonstrating due diligence with FDA software quality requirements. Whether your device is Class I, II or III, the VectorCAST tools provide a dependable and repeatable testing process for your medical device software development. And the VectorCAST coverage and test case pass/fail reports are exportable to regular HTML or text for easy integration into product documentation delivered to the FDA.
FDA Developer Testing Levels
|FDA Developer Testing Levels||Satisfied with VectorCAST|
|Unit (module or component) level testing focuses on the early examination of sub-program functionality and ensures that functionality not visible at the system level is examined by testing. Unit testing ensures that quality software units are furnished for integration into the finished software product.||
|Integration level testing focuses on the transfer of data and control across a program's internal and external interfaces. External interfaces are those with other software (including operating system software), system hardware, and the users and can be described as communications links.||
|System level testing demonstrates that all specified functionality exists and that the software product is trustworthy. This testing verifies the as-built program's functionality and performance with respect to the requirements for the software product as exhibited on the specified operating platform(s).||
According to the FDA General Principles of Software Validation Guidance:
All production and/or quality system software, even if purchased off-the-shelf, should have documented requirements that fully define its intended use, and information against which testing results and other evidence can be compared, to show that the software is validated for its intended use.
Vector Software has developed an extensive software medical device certification kit VectorCAST to meet FDA and IEC 62304:2006 standards.
Further, the FDA document provides a recommended approach to Testing by the Software Developer in section 5.2.5.The various VectorCAST test tools address the software testing referred to in this section and provide an automated capability to satisfy each of the document's requirements.
Section 5.2.5: Testing by the Software Developer
Software testing entails running software products under known conditions with defined inputs and documented outcomes that can be compared to their predefined expectations.
- VectorCAST/C++ supports pass/fail analysis of specificed input data and defined expected results for each test case
An essential element of a software test case is the expected result. It is the key detail that permits objective evaluation of the actual test result.
- With VectorCAST/C++, testers can specify a wide range of valid results for a test including range of values, floating point tolerance, and an exception being raised.
Code-based testing is also known as structural testing or "white-box" testing. It identifies test cases based on knowledge obtained from the source code, detailed design specification, and other development documents. Structural testing is accomplished primarily with unit (module) level testing, but can be extended to other levels of software testing.
- VectorCAST can support both white box and black box testing
Code Coverage Analysis
The level of structural testing can be evaluated using metrics that are designed to show what percentage of the software structure has been evaluated during structural testing. These metrics are typically referred to as "coverage" and are a measure of completeness with respect to test selection criteria. The amount of structural coverage should be commensurate with the level of risk posed by the software.
- Both VectorCAST/C++ and VectorCAST/Cover provide code coverage metrics. These metrics show you what areas of your code have been exercised and more importantly, what areas of the code have not. It is those areas of code that are not exercised that have increased risk of software failure.
VectorCAST Code Coverage Viewer
VectorCAST Path Coverage Report
Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change.
- VectorCAST/Manage allows for full regression testing of all VectorCAST testing at a frequency of your choice, including nightly
Test reports should comply with the requirements of the corresponding test plans.
- VectorCAST/RGW (Requirements Gateway) enables the mapping of requirements to VectorCAST/C++ unit test cases, and lets users push that mapping information back to their Requirements Traceability tool for full analysis. VectorCAST supports bi-directional data transfer with many requirements analysis tools like DOORS, Requisite Pro, Reqtify, and Polarion, as well as Microsoft Excel and Word.
Requirements to Test Case Mapping
Improving Code Quality While Complying with IEC 62304
Any company that wants to sell a medical device in Europe needs to comply with the IEC 62304 standard.The standard provides a list of tasks and activities that support the safe design and maintenance of medical device software. The goal of this is to ensure the software does what is intended without causing any unacceptable risks. Therefore, IEC 62304 relies heavily on Risk Management strategies.
There are several ways that VectorCAST tools can increase code quality and therefore reduce risk, regardless of whether you developing Class A, B or C medical devices:
VectorCAST/Cover provides standalone code coverage analysis during system testing to identify not only what areas of the application are being executed during system testing but more importantly, which areas are not being executed. Those areas have a higher risk of failure. A typical risk mitigation strategy is to increase the tests being performed on the software to achieve 100% code coverage.
VectorCAST/C++ allows engineers to set up both low level unit tests, which test a single function, as well as higher level software module integration tests, which test a logical grouping of functions. These tests consist of specific inputs to stimulate a function and the expected results indicating successful execution of the code based on the inputs. These tests can be run natively, on a target simulator, or on live target hardware. VectorCAST provides pass/fail analysis to show which expected results were achieved, meaning a passing test case, and which were not, a failing test case. Passing 100% of the tests leads to a higher quality product, and when combined with the code coverage metrics that VectorCAST/C++ also offers, can reduce the risk of failure for code that achieves 100% coverage.
The other area where risk is introduced is when the software changes and the changes are not thoroughly analyzed as to their impact on the overal software application. Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change. With VectorCAST/Manage, regression testing is automated and can be set up as frequently as you want, including nightly.
Commercial Test Tool Validation
Software testing tools are frequently used to ensure consistency, thoroughness, and efficiency in the testing of such software products and to fulfill the requirements of the planned testing activities. These tools may include supporting software built in-house to facilitate unit (module) testing and subsequent integration testing (e.g., drivers and stubs) as well as commercial software testing tools. Such tools should have a degree of quality no less than the software product they are used to develop. Appropriate documentation providing evidence of the validation of these software tools for their intended use should be maintained, known as "Intended Use Validation."
VectorCAST provides tool validation in the form of qualification documents. The qualification documents consist of Tool Operational Requirements (TOR) and Tool Qualification Data (TQD). TOR describes requirements for VectorCAST tools and TQD, the associated tests and test results. These documents are built for each project and the tests execute on the same compiler/target/runtime environment used by the customer.
The VectorCAST Medical Device Software Compliance Kit for FDA and IEC 62304 includes all necessary documentation for validating embedded software components used on all classes of medical devices.
The kit includes test procedures, plans, coverage and test case reports necessary for Intended Use Validation, and to demonstrate due diligence with FDA and IEC 62304 software quality requirements.
See the Benefits of VectorCAST with Your Medical Embedded Software Applications
Discover how VectorCAST code coverage analysis tools improve performance in your exact testing environment by registering today for a 30-day, fully-functional version.
You may also contact us to arrange a demo for your project.