Proving Due Diligence for FDA Audits

VectorCAST tools are successfully used by clients demonstrating due diligence with FDA software quality requirements. Whether your device is Class I, II or III, the VectorCAST tools provide a dependable and repeatable testing process for your medical device software development. And the VectorCAST coverage and test case pass/fail reports are exportable to regular HTML or text for easy integration into product documentation delivered to the FDA.

FDA Developer Testing Levels

According to the FDA General Principles of Software Validation Guidance, ‘All production and/or quality system software, even if purchased off-the-shelf, should have documented requirements that fully define its intended use, and information against which testing results and other evidence can be compared, to show that the software is validated for its intended use.’

Vector Software has developed an extensive software medical device certification kit VectorCAST to meet FDA and IEC 62304:2006 standards.

Further, the FDA document provides a recommended approach to Testing by the Software Developer in section 5.2.5.The various VectorCAST test tools address the software testing referred to in this section and provide an automated capability to satisfy each of the document's requirements.

Section 5.2.5: Testing by the Software Developer

Unit Testing

Software testing entails running software products under known conditions with defined inputs and documented outcomes that can be compared to their predefined expectations.

• VectorCAST/C++ supports pass/fail analysis of specificed input data and defined expected results for each test case

An essential element of a software test case is the expected result. It is the key detail that permits objective evaluation of the actual test result.

• With VectorCAST/C++, testers can specify a wide range of valid results for a test including range of values, floating point tolerance, and an exception being raised.

Code-based testing is also known as structural testing or "white-box" testing. It identifies test cases based on knowledge obtained from the source code, detailed design specification, and other development documents. Structural testing is accomplished primarily with unit (module) level testing, but can be extended to other levels of software testing.

• VectorCAST can support both white box and black box testing

Pass/Fail Metrics

Code Coverage Analysis

The level of structural testing can be evaluated using metrics that are designed to show what percentage of the software structure has been evaluated during structural testing. These metrics are typically referred to as "coverage" and are a measure of completeness with respect to test selection criteria. The amount of structural coverage should be commensurate with the level of risk posed by the software.

• Both VectorCAST/C++ and VectorCAST/Cover provide code coverage metrics. These metrics show you what areas of your code have been exercised and more importantly, what areas of the code have not. It is those areas of code that are not exercised that have increased risk of software failure.

VectorCAST Code Coverage Viewer

VectorCAST Path Coverage Report

Regression Testing

Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change. 

• VectorCAST/Manage allows for full regression testing of all VectorCAST testing at a frequency of your choice, including nightly

Requirements Tracing

Test reports should comply with the requirements of the corresponding test plans.

• VectorCAST/RGW (Requirements Gateway) enables the mapping of requirements to VectorCAST/C++ unit test cases, and lets users push that mapping information back to their Requirements Traceability tool for full analysis. VectorCAST supports bi-directional data transfer with many requirements analysis tools like DOORS, Requisite Pro, Reqtify, and Polarion, as well as Microsoft Excel and Word.
  

Requirements to Test Case Mapping

Improving Code Quality While Complying with IEC 62304

Any company that wants to sell a medical device in Europe needs to comply with the IEC 62304 standard.The standard provides a list of tasks and activities that support the safe design and maintenance of medical device software. The goal of this is to ensure the software does what is intended without causing any unacceptable risks. Therefore, IEC 62304 relies heavily on Risk Management strategies.

There are several ways that VectorCAST tools can increase code quality and therefore reduce risk, regardless of whether you developing Class A, B or C medical devices:

VectorCAST/Cover provides standalone code coverage analysis during system testing to identify not only what areas of the application are being executed during system testing but more importantly, which areas are not being executed. Those areas have a higher risk of failure. A typical risk mitigation strategy is to increase the tests being performed on the software to achieve 100% code coverage.

VectorCAST/C++ allows engineers to set up both low level unit tests, which test a single function, as well as higher level software module integration tests, which test a logical grouping of functions. These tests consist of specific inputs to stimulate a function and the expected results indicating successful execution of the code based on the inputs. These tests can be run natively, on a target simulator, or on live target hardware. VectorCAST provides pass/fail analysis to show which expected results were achieved, meaning a passing test case, and which were not, a failing test case. Passing 100% of the tests leads to a higher quality product, and when combined with the code coverage metrics that VectorCAST/C++ also offers, can reduce the risk of failure for code that achieves 100% coverage.

The other area where risk is introduced is when the software changes and the changes are not thoroughly analyzed as to their impact on the overal software application. Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change.  With VectorCAST/Manage, regression testing is automated and can be set up as frequently as you want, including nightly.

Commercial Test Tool Validation

Software testing tools are frequently used to ensure consistency, thoroughness, and efficiency in the testing of such software products and to fulfill the requirements of the planned testing activities. These tools may include supporting software built in-house to facilitate unit (module) testing and subsequent integration testing (e.g., drivers and stubs) as well as commercial software testing tools. Such tools should have a degree of quality no less than the software product they are used to develop. Appropriate documentation providing evidence of the validation of these software tools for their intended use should be maintained, known as "Intended Use Validation."

FDA and IEC 62304 Compliance

VectorCAST provides tool validation in the form of qualification documents. The qualification documents consist of Tool Operational Requirements (TOR) and Tool Qualification Data (TQD). TOR describes requirements for VectorCAST tools and TQD, the associated tests and test results. These documents are built for each project and the tests execute on the same compiler/target/runtime environment used by the customer.

The VectorCAST Medical Device Software Compliance Kit for FDA and IEC 62304 includes all necessary documentation for validating embedded software components used on all classes of medical devices.

The kit includes test procedures, plans, coverage and test case reports necessary for Intended Use Validation, and to demonstrate due diligence with FDA and IEC 62304 software quality requirements.