Clinical evidence requirements according to the IVDR 2017/746: practical tools and references for underpinning clinical evidence of IVD-MDs

Introduction

The new European regulation for in vitro diagnostic medical devices 2017/746 [1] (IVDR) came into force on May 2022 and replaces the directive 98/79/EC (IVDD) [2]. It reinforces the requirements for manufacturers with a view to improving the performance and safety of devices and increasing transparency throughout the industry sector.

Extensions to transition periods have been granted. They depend on the class of the concerned device and the status of its CE marking under IVDD [3]. They are mainly justified by the lack of certified notified bodies (NB) able to deliver the CE marking. Under the IVDD, 8% of the in vitro diagnostic medical devices (IVD-MDs) on the market involved CE marking with control by NB. With the new regulation, the proportion should increase up to 78% [4]. There are questions about the degree of preparation of producers/users of IVD-MDs to meet these stricter obligations, in particular on the point of demonstrating performances. Furthermore, only eight NB are currently certified, compared to about 20 previously. The European Commission wants to be reassuring [5], but the number of NB remains largely insufficient and there is a risk that companies will not be able to market new IVD-MDs [6]. The Medical Device Coordination Group (MDCG) proposed solutions in the MDCG 2022-14 [7] and the European Commission published an amendment in January 2023 providing a phased roll out [8]. IVDR 2017/746 still leads to more threats than opportunities, especially for home-made test, high risk IVD-MDs [9] or for IVD-MDs use outside their approved indications. Therefore, the authors highlighted the effort required to ensure that all these tests can be performed in compliance with the IVDR.

Others point out difficulties of understanding and implementing articles from IVDR, even if the MDCG has published support documents to help with interpretation and despite the existence of international standards [7, 10]. For example, van Deutekom and Haitjema raised the issue that software development is not integrated into the ISO-15189 standard and note a lack of knowledge in medical laboratories on how to deal with software developed in-house, using ISO-15189 in synergy with IEC-62304 [10, 11]. They published recommendations to help the scientific community to facilitate the compliance with IVDR and described three use cases to illustrate their proposed methodology [12].

All IVD-MDs must comply with the new IVDR. This includes up-classification of devices already on the market, CE-marked devices under IVDD, using IVD-MDs outside their approved indications and marking of new devices. This paper aims at helping developers of IVD-MDs (researcher, manufacturer, labs staff) to better understand the key requirements in the IVDR, such as the evaluation of clinical evidence. We also provide references describing how to meet them.

Overview of IVDR content

The IVDR contains 10 chapters and 15 annexes (Supplementary Table 1). It establishes the rules for placing on the market, making available on the market or putting into service IVD-MDs for human use and their accessories in the European Union, and applies to performance studies concerning these devices and their accessories carried out in the Union. A number of new concepts compared to the IVDD are included (near-patient testing devices, companion diagnosis devices, falsified device, kit, common specifications, genetic testing and single-use testing). All manufacturers have to prove that their IVD-MDs comply with the general safety and performance requirements, even for in-house IVD-MDs.

Classification of IVD-MDs is now based on the risk level. Class A corresponds to the least risky devices and includes sample containers or buffer solutions (among others). Self-testing devices with no risk like pregnancy or fertility tests belong to class B. Companion devices, such as those used to screen for cancer, perform prenatal screenings or blood grouping devices are C classified. D is the riskiest class with devices for the detection or diagnosis of life-threatening or transmissible diseases. The definition of the IVD-MDs class conditions the whole CE marking process. For example, class A non-sterile devices can be self-certified. For other classes, conformity assessment involves a NB. For class B and above, a quality system is required. For classes C and D, the device conformity can be proved in two ways: establishing a complete quality management system or a type examination and a “production quality assurance”. Class D devices also involve batch control, but an additional opinion of a reference laboratory is required.

Performance evaluations and interventional studies are reported as an addition to the initial technical or post-marketing surveillance documentation.

Note that the IVDR covers IVD-MDs throughout their life cycle. Consequently, follow-up to ensure post-marketing performance monitoring, vigilance and market surveillance must be performed and the technical documentation regularly updated.

Key steps for bringing IVD-MDs to the EU-market

Throughout the lifecycle of an IVD-MD (see [13] for a detailed flow chart of the certification cycle), performance evaluation is not the first step (Supplementary Figure SF1). Indeed, it is necessary to ensure that the device is covered by the IVD-MD definition. To summarize, an IVD-MD is a MD intended to be used in vitro for the examination of samples from the human body, the purpose being to provide information on the patient’s health status. Specimen containers are also considered as IVD-MDs.

Once the IVD-MD status is identified, the intended use of the IVD-MD must be precisely and carefully defined. The compliance with the essential requirements must then be demonstrated, including performance evaluation for the expected intended use of the IVD-MD. CE mark can be obtained only if all conditions of performance and safety are met. It must be maintained by carrying out post-market surveillance and vigilance.

About clinical evidence

Clinical evidence must be provided during demonstration of performance and more precisely during the demonstration of clinical performance. According to the IVDR, “clinical evidence means clinical data and performance evaluation results, pertaining to a device of a sufficient amount and quality to allow a qualified assessment of whether the device is safe and achieves the intended clinical benefit(s), when used as intended by the manufacturer”.

In other words, clinical evidence demonstrates that the intended clinical benefit(s) and safety is achieved in accordance with the state of the art in the concerned medical field. This clinical evidence is based on three main pillars (Supplementary Figure SF2).

1. Scientific validity proves that the analyte, marker or molecule being studied is relevant to the physiological or pathological condition of interest.

2. Analytical performance is the ability of the device to measure this analyte, marker or molecule accurately and reproducibly. It is strictly a technical performance as no correlation with the targeted pathology is required. It is possible to demonstrate the analytical performance of an IVD-MD with artificial samples. This must demonstrate the accuracy, the measurements reproducibility, the limit of the blank, the analytical specificity and all other technical parameters ensuring the reliability of the measurements.

3. Clinical performance is the ability of an IVD-MD to produce results which correlate with the actual clinical condition, depending on the target population and user. There exists an underlying intention of use that can be highlighted by a performance indicator. In order to demonstrate clinical benefit, dedicated guides depict the general principles of clinical evidence [7, 14].

Two other concepts are important to stakeholders but are addressed differently by the IVDR: clinical benefit and clinical utility. The notion of clinical benefit often appears in the IVDR but no mention of clinical utility is found.

Clinical benefit “means the positive impact of a device related to its function such as that of screening, monitoring, diagnosis or aid to diagnosis of patients, or a positive impact on patient management or public health”. Important, clinical benefit is not identically addressed by MDR and IVDR. While for MD, clinical benefit is measured directly by a positive impact on the patient’s health, for IVD-MDs, the objective is to obtain precise medical information on the patient. The positive impact on the patient is therefore indirect, since the final clinical result depends on other therapeutic options.

Clinical utility is “the likelihood that a test will, by prompting an intervention, result in an improved health outcome” (National Institute of Health). In their article, Horvath et al. defined clinical utility as “Ability of a test to improve health outcomes that are relevant to the individual patient” [15]. The clinical utility clearly targets the patient, whereas the clinical benefit is a broader concept extended to public health but potentially with no direct benefit for the tested patient. For example, screening for COVID 19 allows protecting populations by implementing an adapted public health policy more than improving a patient health outcome. Furthermore, clinical benefit is clearly defined and will not be in the “eyes of the beholder” like clinical utility could be, as explain by Lesko et al. in their interesting issue [16].

Performance evaluation process

Collection of clinical evidence is a part of a performance evaluation process. This process is maintained throughout the lifecycle of the IVD-MD. In addition to the IVDR and other documents [7, 13, 17], ISO 20916 [18] can be used to ensure that the clinical performance study will result in reliable and robust results.

Writing the performance evaluation plan (PEP)

The starting point of the process is the performance evaluation plan (PEP). The PEP identifies the approach, the method to be used and the steps required to generate clinical evidence. Although it remains very global, it describes how to prove that the three above-mentioned pillars are robust. It thus integrates basic elements concerning the IVD-MD (intended use, characteristic, analyte, type of patients, general requirements), more precise elements concerning the evaluation itself (methods, statistical analysis plan, state of the art, benefit/risk ratio) and steps planned for post-marketing performance monitoring (Figure 1A).

Figure 1:

Documentation to be produced for the performance evaluation. The performance evaluation plan is used to plan the entire process (A), while the clinical performance evaluation plan should be thought of as a clinical investigation protocol (B). After the completion of one of the three types of clinical performance studies (C), a clinical performance evaluation report (D) is written. It will be added to the performance evaluation report which also includes the scientific validity and analytical performance reports (E).

To write an appropriate PEP, it is important to define minimum clinical performance levels a new test must reach (Figure 2, 1). Lord et al. propose a 5 steps approach to help achieving this goal [19]. The first step is to define the intended benefits: improving disease outcomes, reducing iatrogenic harm or other benefits (reducing cost for example). The second step is to map current practice. The third step is to propose test role by redrawing the clinical pathway with the new test positioned in: add-on, triage, replacement test. It can be a new pathway (fulfilled an unmet need) if there is no existing test. The fourth step is to link clinical performance requirements to intended benefits such as improving detection, improving rule out or improving test process. The final step is to set minimum acceptable clinical performance levels.

Figure 2:

Roadmap for the collection of clinical evidence for relevant questions, performance assessments and mandatory reporting to comply with IVDR.

Conclusions

The IVDR reinforces the notions of traceability in terms of safety and performance of in vitro devices. This leads to an increase of the documentation to be produced and induces additional administrative constraints. Since the time required to obtain the necessary authorizations can be long (without any guarantee of a favorable outcome), this difficult and time-consuming work should be anticipated.

For existing devices, peer-reviewed literature or published experience with routine diagnostic tests can be an important source of clinical performance data. This is, for existing tests, sufficient in many cases. Review of scientific literature should follow established guidance, like PRISMA for systematic review [24, 25]. Specific tools could be used for assessment of diagnostic accuracy studies [26] or comparative diagnostic test accuracy studies [27]. For others, analytical performance tests should be performed in the same way as before (sensitivity, specificity, accuracy, linearity, etc.). For almost all, NBs also expect data from clinical performance studies with leftover specimens, prospective samples and/or from interventional studies. Study design depends, among others, on objectives, expected performances and intended use. GHTF/SG5/N8:2012 provides different designs and examples [33]. It is also crucial to define the sample size based on predefined clinical criteria such as area under the curve, positive or negative predictive value, or positive and negative likelihood ratio and prevalence [36, 37].

To summarize, a single approach does not exist. The roadmap proposed in this paper, inspired by articles from Lord et al. [19] and from Leeflang and Allerberger [21] aims at helping researchers, manufacturers and laboratories staff asking the relevant questions related to the documents requested in the IVDR (scientific validity report, analytical performance report, clinical performance report). A large number of documents and guidelines describing the IVDR roll out and how to conduct performance studies exist. In this paper, we provide key references which, we believe, will help concerned developers to easily find their way in this complex regulatory environment which is still under construction.