CQ Guide - How to conduct Performance Evaluation for In Vitro Diagnostic (IVD) Devices

Steps in Performance Evaluation Process

The steps in the performance evaluation process typically include the following:

• Establish a performance evaluation plan by defining the objectives and criteria for evaluating performance.
• Demonstration of scientific validity to measure and whether it has been developed using sound scientific principles.
• Demonstration of analytical performance with technical performance test, such as its precision, accuracy, and reproducibility.
• Demonstration of clinical performance by evaluating the clinical performance of the test, such as its sensitivity, specificity, positive and negative predictive values, and receiver operating characteristic (ROC) curve.
• Assessment of the reports for clarity, completeness, as well as the user-friendliness of the interface.
• Demonstration of clinical evidence through tests to validate clinical settings, and confirm if there is sufficient evidence to support its use in clinical practice.
• Performance evaluation report is the final report that summarizes the results of the performance evaluation, including any limitations or recommendations for future improvements.

How to Plan For a Performance Evaluation

A performance evaluation plan is a detailed document that outlines how a medical device will be tested, analyzed, and validated. It is essential to ensure that a medical device performs accurately, reliably, and safely before it is released for use in patients. A well-designed performance evaluation plan should include the following elements:

• Intended purpose describes the intended use of the medical device, including the target population, disease or condition, and clinical setting.
• Characteristics of the device focuses on the physical and technical characteristics of the medical device, including its design, technology, and components.
• Analyte or marker details the specific analyte or marker that the device is intended to measure or detect.
• Certified reference material that will be used to establish the device's metrological traceability.
• Procedure allowing metrological traceability explains the procedure that will be used to establish the device's metrological traceability.
• Target patient groups with indications, limitations, and contraindications
• GSPR requirement that the device must meet to ensure its safety and performance.
• Specification of method that will be used to analyze the device's performance, as well as the methods for examining its analytical or clinical performance and any limitations.
• Description of state of art provides a review of the current state of the art for the type of medical device being evaluated.
• Relevant standards, guidance, common specifications that will be used to evaluate the device's performance.
• Indication and specifications of parameters to be used for the analytical and clinical performance
• Outline of development phases for the device, including the means of determination of validity and performance, indications of milestones, and description of potential acceptance criteria.
• PMPF planning, including any plans for additional testing or monitoring of the device's performance after it is released to the market.
• Justification for not including any of the mentioned elements that were not included in the performance evaluation plan.

Demonstration of the Analytical and Clinical Performance

Analytical performance refers to the ability of a diagnostic test to accurately measure a specific biomarker or parameter. The following are some ways to demonstrate analytical performance:

• Analytical Performance Studies: Analytical performance studies evaluate the accuracy and precision of a diagnostic test. These studies are often conducted in a laboratory setting and involve testing the diagnostic test using known samples with varying concentrations of the biomarker or parameter.
• Comparison to Existing Clinical Standard Practice: In cases where there are no certified reference materials or reference measurement methods available for a novel biomarker or parameter, comparison to existing clinical standard practice can provide evidence for the analytical performance of the diagnostic test.
• Use of Well-documented or Composite Standards: In the absence of comparative methods, the use of well-documented or composite standards can be used as a reference point to evaluate the analytical performance of the diagnostic test.

Clinical performance refers to the ability of a diagnostic test to accurately identify a particular disease or condition in a clinical setting. The following are some ways to demonstrate clinical performance:

• Clinical Performance Studies: It evaluate the accuracy and reliability of a diagnostic test in a clinical setting. These studies often involve testing the diagnostic test on patients with known disease or condition and comparing the results to a gold standard or reference standard.
• Scientific Literature: It provides additional evidence for the clinical performance of a diagnostic test. This includes peer-reviewed articles, systematic reviews, and meta-analyses that report on the diagnostic accuracy and reliability of the test.
• Published Experiences Gained by Routine Diagnostic Testing: The published experiences of clinicians and laboratories using the diagnostic test in routine clinical practice can provide additional evidence for the clinical performance of the test. This includes case reports and clinical case series.

The Performance Evaluation Report

The Performance Evaluation Report is a critical component of the process of bringing a new medical device to market. The report should contain information such as,

Scientific Validity and Analytical and Clinical Performance Report: The scientific validity report should describe the device's scientific principles and its mode of operation. It should demonstrate that the device's underlying technology is sound and that the device performs its intended function accurately and reliably. The analytical and clinical performance report should describe the device's performance characteristics and provide data on its accuracy, precision, sensitivity, and specificity.

Assessments of the Reports Allowing Demonstration of Clinical Evidence: The report should include assessments of the scientific validity, analytical, and clinical performance reports that allow for the demonstration of clinical evidence. The assessments should describe how the data collected demonstrates that the device is safe and effective and can be used in clinical practice.

Justification for the Approach to Gather Clinical Evidence: The report should justify the approach used to gather clinical evidence. The justification should explain how the clinical evidence was gathered, the sample size used, and the methods used to analyze the data.

Literature Search Methodology, Protocol, and Report of Literature Review: The report should describe the methodology used to conduct the literature search, the protocol followed, and the report of the literature review. The literature search should be comprehensive and include relevant studies that demonstrate the device's safety and effectiveness.

Technology on Which Device is Based, Intended Purpose of the Device, Device Performance, and Safety Claims: The report should describe the technology on which the device is based, the intended purpose of the device, and the device's performance and safety claims. The device's intended use and its indications for use should be clearly stated.

Nature and Extent of Scientific Validity, Analytical and Clinical Performance Data Should be Evaluated: The report should evaluate the nature and extent of the scientific validity, analytical, and clinical performance data. The evaluation should determine whether the data collected is sufficient to demonstrate the device's safety and effectiveness.

Clinical Evidence as the Acceptable Performance Against the State of Art-in Medicine: The report should compare the device's clinical evidence to the state of the art in medicine. The comparison should demonstrate how the device performs relative to existing devices or therapies.

Conclusion from PMPF Report: The report should include a conclusion from the Performance Evaluation Report that summarizes the data collected and demonstrates that the device is safe and effective for clinical use. The conclusion should be based on a comprehensive analysis of the scientific validity, analytical and clinical performance, and safety claims of the device.

Recently, ComplianceQuest presented a webinar on “Establishing Clinical Evidence the Right Way for IVD Devices – Performance Evaluation.” This webinar talks about what is performance evaluation, how to plan for this and steps of performance evaluation process.

To know more about how to conduct efficient performance evaluation of your IVD devices, visit: https://www.compliancequest.com/checklist/efficient-performance-evaluation/.