VectorCAST is the Solution for Complete FDA-Compliant Medical Embedded Application Testing

Die moderne Medizin greift in zunehmendem Maße auf elektronische Apparate zurück, um die Lebensqualität von Patienten und die Qualität der medizinischen Versorgung zu verbessern. Als Beispiele seien hier Patientenmonitore, Infusionspumpen, Schrittmacher und Defibrillatoren genannt. All diesen Apparaten gemeinsam ist, dass sie mit eingebetteter Software betrieben werden. Zum Wohl der Patienten müssen sie störungs- und unterbrechungsfrei funktionieren. Umfassende Tests sind unabdingbar, damit die Geräte die strengen FDA-Prüfungen bestehen.

Warum Systemtests zum Erzielen 100 % vorhersagbarer und einwandfreier Ergebnisse nicht ausreichend sind

Zur Gewährleistung eines fehlerlosen und vorhersagbaren Softwareablaufs muss sichergestellt sein, dass der Programmcode von den Systemtests vollständig erfasst wird. Meistens liegt aber die Codeabdeckung bei Systemtests weit unter 100 %. Ein Wert im Bereich 60 bis 70 % ist fast die Regel, da es in kritischen Anwendungen wie medizintechnischen Geräten viele "Grenzfälle" gibt. Die Codebehandlungs-Ausfallsicherheitsbedingungen und andere Fehlerfälle sind bei einem vollständigen Gerätetest unter Umständen nicht stimulierbar.

Die Lösung für vollständige und FDA-konforme Test von eingebetteten medizinischen Anwendungen

VectorCAST/C++ Einfach durchzuführende Systemtests plus Modultests einzelner Funktionen oder Funktionsgruppen, um die Testabdeckung zu vervollständigen.

VectorCAST/Cover Automatische Aufzeichnung, welche Teile einer Anwendung von welchen Systemtests stimuliert werden. Unterstützt Statement- und Branch-Coverage und Modified Condition/Decision Coverage (MC/DC). VectorCAST/MCDC Abdeckung wurde ursprünglich für die Luftfahrtindustrie entwickelt, aber inzwischen haben auch führende Hersteller medizintechnischer Geräte diesen Codeabdeckungsstandard übernommen.

Nachweis der Due Diligence für FDA-Prüfungen

VectorCAST-Werkzeuge werden erfolgreich von Kunden angewendet, die die Erfüllung von FDA-Software-Qualitätsanforderungen nachweisen müssen. Abdeckung und Testfallberichte sind in HTML oder Text exportierbar und lassen sich so problemlos in Produktdokumentationen integrieren. Die VectorCAST-Werkzeuge stellen verlässliche und reproduzierbare Testverfahren für die Softwareentwicklung Ihrer medizintechnischen Geräte (egal ob Class I, II oder III) bereit.

In den "General Principles of Software Validation: Final Guidance for Industry and FDA Staff" (Allgemeine Prinzipien der Software-Validierung, Leitfaden für Industrie und Mitarbeiter der FDA) werden in Abschnitt 5.2.5 ("Testing by the Software Developer") Empfehlungen für die durchzuführenden Tests gegeben. Die VectorCAST-Testwerkzeuge unterstützen die dort erwähnten Softwaretests und umfassen automatisierte Funktionen, mit deren Hilfe jede Anforderung des o.g. Leitfadens erfüllt werden kann. Nachstehend finden Sie ins Deutsche übersetzte Auszüge aus Abschnitt 5.2.5 (jeweils kursiv gedruckt) und Angaben zu den Einsatzmöglichkeiten der VectorCAST-Werkzeuge:

5.2.5 Vom Softwareentwickler durchzuführende Tests

...Bei Softwaretests müssen Softwareprodukte unter bekannten Bedingungen ablaufen, mit definierten Eingaben und dokumentierten Ausgaben, die sich mit den vorgegebenen Erwartungen vergleichen lassen.

...Ein wesentliches Element eines Software-Testfalls ist das erwartete Ergebnis. Hier handelt es sich um eine wesentliche Information, die eine objektive Auswertung des Testergebnisses ermöglicht.

...Codebasiertes Testen wird auch strukturelles Testen oder "White-Box"-Testen genannt. Diese Begriffe bezeichnen Testfälle, die auf vom Quellcode, Entwurfsspezifikationen und anderen Entwicklungsdokumenten abgeleiteten Kenntnissen basieren. Strukturelles Testen erfolgt meistens in Form von Modultests, aber auch andere Softwaretestebenen sind möglich.

Codeabdeckung

...Die Ebene der strukturellen Tests kann mit Metriken bewertet werden, die zeigen sollen, welcher Prozentsatz der Softwarestruktur ausgewertet wurde. Diese Metriken werden meistens als "Abdeckung" bezeichnet und sind ein Maß für die Vollständigkeit bezüglich der Testauswahlkriterien. Das Ausmaß der strukturellen Abdeckung sollte in einem angemessenen Verhältnis zu den möglichen Risiken der Software stehen.

Regressionstests

Regressionsanalyse und -tests werden durchgeführt um sicherzustellen, dass eine Modifikation keine Probleme in einem anderen Teil der Software zur Folge hat. Unter einem Regressionstest versteht man die Wiederholung aller Testfälle, die ein Programm vorher korrekt ausgeführt hat, und der Vergleich des Ist-Ergebnisses mit dem Soll-Ergebnis (des vorherigen Tests), um Nebenwirkungen einer Änderung der Software aufzuspüren.

Um eine strenge und gründliche Prüfung eines Softwareprodukts sicherzustellen, werden Entwicklungstests in mehrere Ebenen unterteilt. Eine mögliche Gliederung sind zum Beispiel Modul-, Integrations- und Systemtests.

...Testverfahren, Testdaten und Testergebnisse sind so zu dokumentieren, dass objektive Pass/Fail-Entscheidungen getroffen werden können. Es muss möglich sein, sie nach dem Testablauf zu prüfen, damit eine objektive Entscheidungsfindung durchführbar ist. Und sie müssen in einem nachfolgenden Regressionstest erneut verwendbar sein. Bei einem Test auftretende Fehler sind zu protokollieren, zu klassifizieren, zu überprüfen und zu beheben, bevor die Software freigegeben werden kann. Während eines Entwicklungs-Lebenszyklus erfasste und analysierte Softwarefehlerdaten können genutzt werden, um die Eignung des Softwareprodukts zur Freigabe für kommerzielle Vermarktung festzustellen.

Rückverfolgbarkeit von Anforderungen (Anforderungsverfolgung)

...Testberichte müssen die Anforderungen der entsprechenden Testpläne erfüllen.

VectorCAST/Requirements Gateway erlaubt den Datenaustausch zwischen einem Anforderungsmanagement-Werkzeug und den VectorCAST-Testwerkzeugen. Mit diesen Informationen können Sie problemlos Berichte erstellen, aus denen hervorgeht, dass allen Anforderungen Testfälle zugeordnet sind.

Validierung kommerzieller Testwerkzeuge

Software-Testwerkzeuge werden häufig eingesetzt, wenn bei den Tests solcher Softwareprodukte Übereinstimmung, Gründlichkeit und Effizienz sichergestellt werden soll, und um die Anforderungen der geplanten Testaktivitäten zu erfüllen. Solche Werkzeuge können selbst entwickelte, unterstützende Software umfassen, die die Modultests und anschließenden Integrationstests (z. B. Treiber und Stubs) ermöglichen, ebenso wie kommerzielle Software-Test-Tools. Das Qualitätsniveau der Werkzeuge darf nicht unter dem des Softwareprodukts liegen, bei dessen Entwicklung sie zur Anwendung kommen. Die Validierung der Softwarewerkzeuge für ihren vorgesehenen Bestimmungszweck ist mittels entsprechender Unterlagen nachzuweisen.

Bei VectorCAST erfolgt die Validierung über die Zulassungsdokumente "Tool Operational Requirements" (TOR) und "Tool Qualification Data" (TQD). TOR beschreibt die Anforderungen für VectorCAST-Werkzeuge und TQD die zugehörigen Tests und Testergebnisse. Diese Dokumente werden für jedes Projekt erstellt, und die Tests werden in der gleichen Compiler-/Ziel-/Laufzeit-Umgebung ausgeführt, wie sie auch der Kunde hat.

Proving Due Diligence for FDA Audits

VectorCAST tools are successfully used by clients demonstrating due diligence with FDA software quality requirements. Whether your device is Class I, II or III, the VectorCAST tools provide a dependable and repeatable testing process for your medical device software development. And the VectorCAST coverage and test case pass/fail reports are exportable to regular HTML or text for easy integration into product documentation delivered to the FDA.

FDA Developer Testing Levels

FDA Developer Testing Levels Satisfied with VectorCAST
Unit (module or component) level testing focuses on the early examination of sub-program functionality and ensures that functionality not visible at the system level is examined by testing. Unit testing ensures that quality software units are furnished for integration into the finished software product.
  • VectorCAST/C++ (unit test)
  • VectorCAST/RSP (target testing)
  • VectorCAST/Manage (regression test)
  • VectorCAST/RGW (requirements tracing)
Integration level testing focuses on the transfer of data and control across a program's internal and external interfaces. External interfaces are those with other software (including operating system software), system hardware, and the users and can be described as communications links.
  • VectorCAST/C++ (unit test)
  • VectorCAST/RSP (target testing)
  • VectorCAST/Manage (regression test)
  • VectorCAST/RGW (requirements tracing)
System level testing demonstrates that all specified functionality exists and that the software product is trustworthy. This testing verifies the as-built program's functionality and performance with respect to the requirements for the software product as exhibited on the specified operating platform(s).
  • VectorCAST/Cover
  • VectorCAST/RGW

According to the FDA General Principles of Software Validation Guidance, ‘All production and/or quality system software, even if purchased off-the-shelf, should have documented requirements that fully define its intended use, and information against which testing results and other evidence can be compared, to show that the software is validated for its intended use.’

Vector Software has developed an extensive software medical device certification kit VectorCAST to meet FDA and IEC 62304:2006 standards.

Further, the FDA document provides a recommended approach to Testing by the Software Developer in section 5.2.5.The various VectorCAST test tools address the software testing referred to in this section and provide an automated capability to satisfy each of the document's requirements.

Section 5.2.5: Testing by the Software Developer

Unit Testing

Software testing entails running software products under known conditions with defined inputs and documented outcomes that can be compared to their predefined expectations.

  • VectorCAST/C++ supports pass/fail analysis of specificed input data and defined expected results for each test case

An essential element of a software test case is the expected result. It is the key detail that permits objective evaluation of the actual test result.

  • With VectorCAST/C++, testers can specify a wide range of valid results for a test including range of values, floating point tolerance, and an exception being raised.

Code-based testing is also known as structural testing or "white-box" testing. It identifies test cases based on knowledge obtained from the source code, detailed design specification, and other development documents. Structural testing is accomplished primarily with unit (module) level testing, but can be extended to other levels of software testing.

  • VectorCAST can support both white box and black box testing


Pass/Fail Metrics

 

Code Coverage Analysis

The level of structural testing can be evaluated using metrics that are designed to show what percentage of the software structure has been evaluated during structural testing. These metrics are typically referred to as "coverage" and are a measure of completeness with respect to test selection criteria. The amount of structural coverage should be commensurate with the level of risk posed by the software.

  • Both VectorCAST/C++ and VectorCAST/Cover provide code coverage metrics. These metrics show you what areas of your code have been exercised and more importantly, what areas of the code have not. It is those areas of code that are not exercised that have increased risk of software failure.


VectorCAST Code Coverage Viewer


VectorCAST Path Coverage Report

 

Regression Testing

Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change. 

  • VectorCAST/Manage allows for full regression testing of all VectorCAST testing at a frequency of your choice, including nightly

Requirements Tracing

Test reports should comply with the requirements of the corresponding test plans.

  • VectorCAST/RGW (Requirements Gateway) enables the mapping of requirements to VectorCAST/C++ unit test cases, and lets users push that mapping information back to their Requirements Traceability tool for full analysis. VectorCAST supports bi-directional data transfer with many requirements analysis tools like DOORS, Requisite Pro, Reqtify, and Polarion, as well as Microsoft Excel and Word.


Requirements to Test Case Mapping

Improving Code Quality While Complying with IEC 62304

Any company that wants to sell a medical device in Europe needs to comply with the IEC 62304 standard.The standard provides a list of tasks and activities that support the safe design and maintenance of medical device software. The goal of this is to ensure the software does what is intended without causing any unacceptable risks. Therefore, IEC 62304 relies heavily on Risk Management strategies.

There are several ways that VectorCAST tools can increase code quality and therefore reduce risk, regardless of whether you developing Class A, B or C medical devices:

VectorCAST/Cover provides standalone code coverage analysis during system testing to identify not only what areas of the application are being executed during system testing but more importantly, which areas are not being executed. Those areas have a higher risk of failure. A typical risk mitigation strategy is to increase the tests being performed on the software to achieve 100% code coverage.

VectorCAST/C++ allows engineers to set up both low level unit tests, which test a single function, as well as higher level software module integration tests, which test a logical grouping of functions. These tests consist of specific inputs to stimulate a function and the expected results indicating successful execution of the code based on the inputs. These tests can be run natively, on a target simulator, or on live target hardware. VectorCAST provides pass/fail analysis to show which expected results were achieved, meaning a passing test case, and which were not, a failing test case. Passing 100% of the tests leads to a higher quality product, and when combined with the code coverage metrics that VectorCAST/C++ also offers, can reduce the risk of failure for code that achieves 100% coverage.

The other area where risk is introduced is when the software changes and the changes are not thoroughly analyzed as to their impact on the overal software application. Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change.  With VectorCAST/Manage, regression testing is automated and can be set up as frequently as you want, including nightly.

 

Commercial Test Tool Validation

Software testing tools are frequently used to ensure consistency, thoroughness, and efficiency in the testing of such software products and to fulfill the requirements of the planned testing activities. These tools may include supporting software built in-house to facilitate unit (module) testing and subsequent integration testing (e.g., drivers and stubs) as well as commercial software testing tools. Such tools should have a degree of quality no less than the software product they are used to develop. Appropriate documentation providing evidence of the validation of these software tools for their intended use should be maintained, known as "Intended Use Validation."

FDA and IEC 62304 Compliance

VectorCAST provides tool validation in the form of qualification documents. The qualification documents consist of Tool Operational Requirements (TOR) and Tool Qualification Data (TQD). TOR describes requirements for VectorCAST tools and TQD, the associated tests and test results. These documents are built for each project and the tests execute on the same compiler/target/runtime environment used by the customer.

The VectorCAST Medical Device Software Compliance Kit for FDA and IEC 62304 includes all necessary documentation for validating embedded software components used on all classes of medical devices.

The kit includes test procedures, plans, coverage and test case reports necessary for Intended Use Validation, and to demonstrate due diligence with FDA and IEC 62304 software quality requirements.

 

I would like to personally thank David Pierson for his hard work and dedication in helping us here at Northrop...Read More »

Fernando Anguiano, Senior Staff Software Engineer, Northrop Grumman Aerospace Systems

VectorCAST matches our development paradigm exactly. Therefore the tool was simple to set up and integrate into our development process....Read More »

Sven Natus, Manager of Software Development, Fujitsu Semiconductor Europe

The capabilities of VectorCAST for unit/integration test and code coverage perfectly complement our existing static analysis solutions. Coupled with our...Read More »

Neil Langmead, Chief Operating Officer, Emenda Software

Vector Software’s VectorCAST product suite is now more integrated with TI’s Code Composer Studio IDE than ever before, offering users...Read More »

John Stevenson, Product and Applications Manager, Texas Instruments

Our confidence that VectorCAST will provide for our unit test needs is quite high and went beyond expectations.Read More »

Ronald L. Martinez, Information Technology, Raytheon Company

System test alone is ineffective in enabling organizations to achieve customer demanded quality and meet market opportunity windows. To achieve...Read More »

Axel Kehlenbeck, Sr. Analyst, Sentry Technology Group

As part of our selection process we evaluated several freeware and commercially available off the shelf applications. We chose VectorCAST...Read More »

Helen Muth, Senior Director of Engineering, MEI Group

The VectorCAST testing environment is an ideal complement to the static analysis capabilities available in GNAT Pro, together providing a...Read More »

Cyrille Comar, Managing Director, AdaCore

We evaluated a number of test tools and were impressed with the VectorCAST automated solution. Using VectorCAST/C++, we have significantly...Read More »

Julieta Catania, Quality Manager, Tecme S.A.

The support we have received from Vector Software has been outstanding. Vector Software’s trainers and support staff consist of highly...Read More »

Stanislav Fligl, Core Team Leader, ŠKODA ELECTRIC

We are delighted that Vector Software has integrated its industry-leading VectorCAST suite with AdaCore’s GNAT Pro Safety Critical product for...Read More »

Jamie Ayre, Marketing Director, AdaCore

I have been using VectorCAST C/C++ on embedded realtime projects for over 3 years. Creating unit tests has been greatly...Read More »

Alan Yorinks, Software Engineer, ITT Geospatial Systems

We were impressed with Vector Software from the outset. VectorCAST provided excellent ease of use and functionality.Read More »

Chris Brown, Vice President of Engineering, Athena Technologies

Vector Software engineers were extremely helpful in supporting our goal to achieve DO-178B certification in short amount of time....Read More »

Pauline Ng, GM Engineering and Technologies, Flight Focus

VectorCAST is a very important component in our embedded software development process allowing us to detect defects early and assure...Read More »

Sven Natus, Manager of Software Development, Fujitsu Semiconductor Europe

VectorCAST is unique in that it provides us with the ability to increase the reliability and quality of our flight...Read More »

Wayne Young, Software Principal Engineer, Honeywell ASCa Inc.

Our leading solutions for Requirements Management, Quality Assurance, and ALM, optimally complement Vector Software’s advanced tools for automated software testing...Read More »

Michael Carey, President, Polarion Software

VectorCAST is the premier dynamic software analysis tool for embedded applications. We are very pleased that our partner, Vector Software,...Read More »

Mats Ullström, Product Director, IAR Systems

Key to our selection of VectorCAST was that it offers a single software engineering platform that can manage process complexity...Read More »

Stefan Sauerer, MBtech

The open interfaces of VectorCAST give us the necessary flexibility to adapt VectorCAST to all needs in all our development...Read More »

Sven Natus, Manager of Software Development, Fujitsu Semiconductor Europe

VectorCAST made our job a lot easier.Read More »

Taeyoung Noh, Quality Assurance Manager NEOS Project, MDS Technology

The decision to use Vector Software's automated solution is driven by our need to achieve the highest levels of software...Read More »

Fábio Assis, Director, XMobots Robotic Systems

Our project requires compliance with DO-178B Level B, which will require stringent testing activities. We needed the best suite of...Read More »

Vladimir Baburov, Director, VNIIRA-Navigator

Automation in testing is typically considered in the context of meeting safety or regulatory requirements. VectorCAST supports our goal of...Read More »

Arun Devaraj, Senior Software Manager, Visteon Electronics

Vector Software’s integration with VxWorks Cert Platform further extends VectorCAST’s support for the comprehensive capabilities of the Wind River Workbench...Read More »

Joe Wlad, Senior Director of Product Management, Wind River

Although no hard cost saving metrics were generated comparing the use of VectorCAST/Ada vs. testing with other tools or testing...Read More »

Mark Hall, Test Consultant, Lockheed Martin

We are very pleased to work with Vector Software. The VectorCAST integration with FreeRTOS provides a engineers with a professional-grade...Read More »

Richard Barry, Director, Real time Engineers, Ltd.

We are enthusiastic about the integration and partnership with such a high caliber test automation company such as Vector Software....Read More »

Regg Struyk, Product Manager, Polarion Software

With VectorCAST we are able to quickly apply state of the art unit testing...Read More »

Yves Schenker, Embedded Software Engineer, Helbling Technik

With its broad support for target compilers and integrated testing capabilities the VectorCAST applications fit well in our current and...Read More »

Sven Natus, Manager of Software Development , Fujitsu Semiconductor Europe

Vector Software's VectorCAST embedded software testing platform was identified as the ideal solution for developing critical software in compliance with...Read More »

Dr. Stefan Kaneff, Managing Director, EUROS Embedded Systems

The right solution to provide our customers products with a superior level of quality.

Giuseppe Todaro, FW/SW Global Technical Leader, Electrolux

Vector Software’s VectorCAST is the leading automated software test tool for safety critical embedded systems, targeting mil/aero, medical and transportation....Read More »

Dan Mender, Vice President Business Development, Green Hills Software

We evaluated VectorCAST based on very specific criteria that included the vendors’ experience in safety-critical embedded software testing and the...Read More »

Michael Dalpiaz, Development Manager, Imaging Systems, Sirona Dental

We decided to adhere to all the rules within MISRA and the QA.C tool made this extremely easy to achieve....Read More »

Lee Jaques, Senior Software Engineer, Marshall Aerospace

Trinity is committed to bringing the most innovative and advanced technologies in safety critical software development. We are convinced that...Read More »

Jeff Chen, Vice President, Trinity Technologies

Big Thanks for this extra-ordinary service!!!Read More »

Ronald L. Martinez, Information Technology, Raytheon Company

With the time savings introduced by VectorCAST, final software development phases could be reached early, reducing the total project costs....Read More »

Mauricio Ribeiro Barbieri, Software Engineer Manager, AEL Sistemas, A.S.

We evaluated many embedded software testing tools during this process. We chose VectorCAST for its robust features and ease of...Read More »

Pauline Ng, GM Engineering and Technologies, Flight Focus

Overall, we reduced our testing effort 83% by using VectorCAST.Read More »

Taeyoung Noh, Quality Assurance Manager NEOS Project, MDS Technology

The combined offering delivers an unparalleled solution for ensuring quality, reliability, and compliance in the development of critical software systems....Read More »

Michael Carey, President, Polarion Software

Vector Software provides one of a kind capabilities to address the challenges of testing embedded software applications. Representing Vector Software...Read More »

Uri Farkash, Director of EDA Operations, Advanced Semiconductor Technologies

We carefully evaluated a number of tools and we were very impressed with the level of automation provided by the...Read More »

Steven Lincoln, C-130 Software Development Manager, Lockheed Martin Aeronautics

We are excited to partner with Vector Software in offering developers a powerful, fully integrated solution for embedded systems development....Read More »

William E. Lamie, President, Express Logic, Inc.

Our test tool evaluation team considered a number of other applications for software testing. We ultimately chose VectorCAST for its...Read More »

Larry Rogers, Avionics Software Manager, Aurora Flight Sciences

The VectorCAST tools have greatly reduced the time it takes to fully test our code. VectorCAST/Manage allows us to...Read More »

Stanislav Fligl, Core Team Leader, ŠKODA ELECTRIC

Vector Software has a proven record of developing the most advanced dynamic software testing solutions that address the real needs...Read More »

Neil Langmead, Chief Operating Officer, Emenda Software

Überzeugen Sie sich von den Vorteilen von VectorCAST für Ihre eingebetteten Medizintechnik-Softwareanwendungen

Finden Sie selbst heraus, wie sich die VectorCAST-Codeabdeckungs-Analysewerkzeuge auch in Ihrer Testumgebung positiv auswirken. Fordern Sie noch heute eine 30 Tage ablauffähige Vollversion an. Gerne können Sie sich auch für unserWebinar über Software-Tests für eingebettete Systeme anmelden, oder wir arrangieren eine projektspezifische Präsentation für Sie