Interlaboratory variation for NT-proBNP among Swedish laboratories in an external quality program 2011–2021

Introduction

An estimation of 26 million individuals in the industrialized world suffer from acute or chronic heart failure (HF) and this number is expected to grow due to increased life expectancy [1, 2]. HF is a challenge for society because of significant mortality and related health care costs [3]. The 76-amino acid N-terminal pro-B-type natriuretic peptide (NT-proBNP) is a hormonally inactive protein originating from the precursor hormone proBNP synthesised by cardiomyocytes in response to intracardiac pressure changes [4]. HF causes intracardiac pressure changes due to tissue stretching and increased volume overload and is associated to elevated NT-proBNP levels in the blood [3, 5, 6]. NT-proBNP can be used for diagnosing, and prognostic evaluation of HF [7, 8]. However, studies on the potential use of NT-proBNP as a biomarker monitoring HF therapy have given conflicting results [9]. Echocardiography on the other hand, is a more labour intensive and expensive method to perform compared to analysing NT-proBNP, but may be used for diagnostic, prognostic, and monitoring purposes in patients with HF [10, 11]. The use of both NT-proBNP and echocardiography for investigating suspected HF is recommended by the European Society of Cardiology (ESC) and this is because they provide different, but complementary and additive pathophysiological and clinical information [9].

NT-proBNP is routinely measured by commercially available immunological methods with antibodies toward different epitopes [12]. These immunoassays might be affected by different forms of glycosylated or fragmented proBNP, which in turn may lead to varying quantifications of NT-proBNP [7, 12], [13], [14], [15], [16]. Previous data indicate that existing methods measure at different levels [17]. Thus, there is a need for a universal reference standard available for NT-proBNP. The International Federation of Clinical Chemistry and Laboratory (IFCC) is working towards harmonization between the methods and has recommended a CV%<10 as a performance goal for NT-proBNP assays [13, 18].

The NT-proBNP method from Roche was the first method suitable for automated large instruments to be introduced in Sweden and it has been followed by methods from Siemens and Abbott [13]. Different NT-proBNP cut-off values for ruling out or classifying HF as unlikely, have been suggested by international guidelines [9, 19]. The guidelines from ESC have set the upper limit of normal NT-proBNP concentrations to <125 ng/L in chronic conditions and to <300 ng/L in acute conditions [9]. The British National Institute for Health and Care Excellence (NICE) recommends <400 ng/L as a cut-off for chronic HF [19]. Apart from having different cut-off values for ruling out HF, there also exist several different context dependent cut-off values for ruling in HF, which makes the situation of evaluating a single NT-proBNP value in a clinical setting complex [9, 19].

The cut-off values for NT-proBNP recommended in international guidelines for HF are often based on Roche’s methods and due to lack of comparative studies it is not yet fully known to what extent and how the quantified values of NT-proBNP from manufacturers such as Roche, Siemens, and Abbott are interconvertible [20], [21], [22], [23], [24], [25], [26]. For instance, the ESC cut-off value of 125 ng/L is based on several studies using Roche [21, 22], but also a radioimmunoassay (RIA) based study [23]. Similarly, the ESC cut-off value of 300 ng/L is based on studies containing results from Roche methods [24], [25], [26]. One comparative study showed that one Siemens method, although not a method among any of the methods included in this study, resulted in 17% higher NT-proBNP values than the Roche method [27]. The consequence of one method constantly yielding higher NT-proBNP results could lead to erroneous diagnostic decisions, as well as unnecessary investigations with echocardiography.

The aim of the present study was to investigate if there were any variations in measured levels of NT-proBNP between the different instrument platforms in the results from the Swedish proficiency testing organisation Equalis (Uppsala, Sweden) during 2011–2021 [28].

Materials and methods

During 2011–2021 Equalis conducted 3–6 rounds per year for a total of 53 rounds of its NT-proBNP external quality assurance (EQA) program. Plasma samples were sent out, with a mean of 46 participating Swedish laboratories per round (range 24–59 laboratories). The number of participants increased during the period studied, and totally 64 unique Swedish laboratories participated, although not all of them participated at the same time. A few foreign laboratories also participated in the EQA programs, but these data were excluded from this study. Each of the 53 NT-proBNP rounds, apart from two rounds 2012–2013, included two plasma samples (total n=104) at different levels, where the lower sample got categorized to the lower range (27–777 ng/L; median=141 ng/L) of NT-proBNP values and the higher sample got categorized to the higher range (181–10,740 ng/L; median=667 ng/L).

The samples were prepared for each round separately and consisted of pooled surplus plasma from anonymized patient samples from Uppsala University Hospital. Some samples were diluted with anonymized surplus plasma from healthy blood donors to achieve desired levels of NT-proBNP. The plasma samples had a 0.8 mL volume and were stored at minus 80 °C before being distributed to the participating laboratories in 2 mL plastic tubes without any additives. From the start of the NT-proBNP EQA program in 2011 until August 2021 the plastic tubes were manufactured by the Sarstedt Group (Nümbrecht, Germany) and thereafter the same type of plastic tubes was manufactured by LVL Technologies (Crailsheim, Germany).

In the present study, a total of 5,919 NT-proBNP results were available. This number does not include obvious written mistakes with unreasonable levels of NT-proBNP that were pre-classified as extreme outliers by Equalis and excluded before survey reports were generated. Additionally, four other results from the first and fourth round of 2012 were excluded due to a high suspicion of a mix up of low and high results in the original reports from the participants.

The participating laboratories reported NT-proBNP levels as raw data to Equalis, thus no data was rounded off. Adjusted values, e.g. factorized, were not allowed in the report as per instructions from Equalis. Since only one participating laboratory was using Ortho Vitros 3600, all these 92 NT-proBNP values were excluded. Also, two odd results from ‘Radiometer’ were excluded. Siemens Dimension Xpand Plus was only used by one laboratory during 2012–2013 and these 13 results were excluded. The remaining 5,808 results were included in the calculations.

Laboratories with multiple instruments had the option to report results for each unique instrument. Most of these laboratories sent in two results. One laboratory occasionally sent in three results starting in 2013 and this increased to two laboratories in 2020 and to three laboratories in 2021. One laboratory occasionally reported four results in some rounds during 2013–2015 and 2017. These instances when one laboratory sent in multiple results originating from separate instruments of the same sort in the same round were treated just as unique results, both in the original survey reports from Equalis and in our calculations. Thus, 19% of the results in this study come from laboratories that reported multiple results.

As there is no internationally accepted reference calibrator or procedure [17], Equalis could not establish a target value for NT-proBNP samples. The reported results were categorized into four groups according to the manufacturer. The Abbott group included: Architect ci 4100, Architect ci 8200, Architect i 1000, Architect i 2000, Alinity c, and Alinity i. The Roche group included: Elecsys 2010, Modular E, Cobas e402, Cobas e411, Cobas e601, Cobas e602, and Cobas e801. For Siemens, the methods were divided into two separate groups. The Stratus instruments designed for near-patient (Stratus CS and Stratus CS 200) testing tended to yield lower results compared to other methods regardless of manufacturer, while the other group with larger Siemens instruments tended to yield higher results. This Siemens ICA (Immulite, Centaur, Atellica) group included Advia Centaur XP, Advia Centaur XPT, Atellica IM Analyzer 1300, Atellica IM Analyzer 1600, Immulite 2000, and Immulite 2500 (Table 1). The Roche and the Siemens Stratus groups participated in every round, while Siemens ICA group participated in every round apart from the fifth low level round in 2019. The Abbott group has participated in every round since their debut in the third round of 2018.

Statistical calculations were performed using MedCalc version 14.8.1 (MedCalc Software, Ostend, Belgium) for Figure 5B and Excel 2019 (Microsoft, Seattle, WA, USA) for everything else. The coefficient of variation (CV; ratio of the standard deviation (SD) divided by the mean value), was used to express the precision and repeatability of an assay. Median values were used instead of mean values since the majority of the program rounds (85 out of 104 rounds) contained data which were not normally distributed according to the Shapiro-Wilk test of normality (data not shown). All data presented has been calculated from raw data and is presented differently compared to the EQA programs.

Results

Manufacturers in the Equalis NT-proBNP EQA program

When the Equalis NT-proBNP EQA program started in 2011 the Roche method was used for 71% of the reported results (total n 2011=195). This proportion peaked at 79% in 2016 (total n 2016=400) and then showed a decreasing trend until reaching its lowest share at 50% in 2021 (total n 2021=754). The Siemens methods combined (ICA and Stratus) contributed 20–29% of the results during 2011–2018, but below 20% since 2019. The Siemens ICA group consisted of only Immulite results from 2011 to 2016 until Centaur was introduced in 2017. In 2020, there were no longer any Immulite results in the Siemens ICA group which by then consisted of Centaur and Atellica, which started to participate the same year. The Abbott method was introduced in 2018 (total n 2018=756), initially with a share of 6% and increasing to 32% in 2021 (Figure 1).

Figure 1:

Share in percent of manufacturer-specific results in the Equalis NT-proBNP EQA program 2011–2021.

Manufacturer median NT-proBNP values compared to total median NT-proBNP values

Total median NT-proBNP was calculated for low and high samples for each round. Figure 2A and B shows how much each manufacturer deviated from the consensus median. Roche, who provided most of the raw data, was closest to the total median in 49 of the 52 low sample rounds (ratio=0.91–1.02) and 49 of the 52 high sample rounds (ratio=0.95–1.02). Siemens ICA had the highest median results in 40 high sample rounds (ratio=0.90–1.45) and 33 low sample rounds (ratio=0.89–1.58). Siemens Stratus had lower median values compared to the total median values in 35 low sample rounds (ratio=0.54–1.25) and in 33 high sample rounds (ratio=0.83–1.25). Abbott was above or equal to the total median in all their 21 high sample rounds (ratio=1.00–1.13) and in 19 of 21 low sample rounds (ratio=0.97–2.48, second highest median ratio=1.41). In Abbott’s first low sample round in 2018, their only two results were the two highest results of that round, thus making Abbott score 2.48 times above the total median (excluded from Figure 2B).

Figure 2:

Ratio for manufacturer-specific deviation from total median (y-axis) for each round (x-axis). (A) High sample deviations. (B) Low sample deviations.

The most recent data from 2019 to 2021 when all four groups participated, show that Roche (ratio=0.99) and Siemens Stratus (ratio=0.94) were below the total median for high samples, while Abbott (ratio=1.05) and Siemens ICA (ratio=1.20) were above. Similarly, Roche (ratio=0.99) and Siemens Stratus (ratio=0.90) were below the rounds’ total median for low samples, while Abbott (ratio=1.07) and Siemens ICA (ratio=1.16) were above.

The years with the lowest deviations from the consensus medians were 2015 and 2016 when total CV for low and high samples were below 13% (Figure 3).

Figure 3:

Total CV in percent (y-axis) for each round (x-axis), high and low samples.

Manufacturer CV for each EQA program round

For the high EQA samples the manufacturer-specific CVs for each round usually amounted to 2–8%. There was a tendency for the Siemens ICA group to yield higher CVs (Figure 4A). The low samples exhibited slightly higher CVs, usually in the range 2–10% (Figure 4B).

Figure 4:

CV in percent for each manufacturer (y-axis) for each round (x-axis). (A) High sample rounds. (B) Low sample rounds.

CV% in comparison to level of NT-proBNP

The total CV% for each low or high sample from each round ranged from 4 to 27%. Total CV per dispatched sample was usually in the range of 5–10%. However, when the consensus median for NT-proBNP was below 200 ng/L the total CV varied more, and frequently reached levels above 10%. The relation between total CV and total median NT-proBNP value for each plasma sample are shown in Figure 5A, while Figure 5B shows this relation in the NT-proBNP interval 0–500 ng/L which are close to the commonly used medical decision limits.

Figure 5:

Total CV in percent (y-axis) and total median NT-proBNP measured in ng/L (x-axis) for each plasma sample sent out. (A) Total CV% for all values of NT-proBNP. (B) Analysis showing the curvilinear inter-relationship between total CV% values and NT-proBNP values in a selected interval of 0–500 ng/L with a 95% confidence interval.

Manufacturer-specific total CVs are visualized for NT-proBNP levels in the range 0–500 ng/L, including the most commonly used medical decision limits, i.e. 125, 300 and 400 ng/L. Almost all of these manufacturer-specific CVs managed to accomplish a CV below 10% (Figure 6). Siemens Stratus had a median CV of 4%, while Abbott and Roche both had a median CV of 5%, and Siemens ICA had a median CV of 6% when looking at all results within this range.

Figure 6:

Total CV in percent for each manufacturer (y-axis) and their median NT-proBNP in a selected interval of 0–500 ng/L (x-axis) for each plasma sample with a statistical analysis of the manufacturer-specific inter-relationship between total CV% and median NT-proBNP.

Discussion

During the period studied the number of participating laboratories increased considerably. Initially, Roche dominated, representing 70–80% of the participants up to 2017. However, from 2018 this market share declined, and was about 50% in 2021. Thus, Roche data will have a high impact of the consensus total medians for all rounds, particularly up to 2017.

Consensus median values for each round were calculated to compare the measured levels for different manufacturers. During the first period 2011–2016 Roche virtually defined the consensus median. The two Siemens groups stayed relatively close to each other but started to diverge from 2017 with Siemens Stratus usually yielding the lowest results, and Siemens ICA the highest compared to consensus. This change coincided with the period when the Siemens ICA group started to get most of its results from Advia Centaur instead of the previously dominating Immulite. Overall, the years with the least variation between manufacturers were 2015 and 2016, when Roche had a high market share, and the new methods from Siemens and Abbott had not yet been introduced.

However, after 2016 results started to diverge. Despite a decreasing market share Roche stayed relatively close to the consensus median during 2019–2021. During this period Siemens ICA yielded higher readings, particularly at high levels of NT-proBNP. Abbott exhibited an intermediate level. Also, there is a large discrepancy between Siemens ICA and Siemens Stratus, although these data are less reliable since there were few participants left in the Stratus group in 2021. Thus, after the introduction of new manufacturers and methods in 2017 the agreement between manufacturers declined.

When looking at EQA rounds with samples for NT-proBNP levels in the range of 100–500 ng/L, total CV varied from 4 to 13%. Thus, most rounds reached the recommendation from the IFCC Committee on Clinical Applications of Cardiac Bio-Markers of a total CV<10% [14] at levels for ruling out HF in acute or chronic settings. In the 100–500 ng/L range, Siemens ICA tended to have the highest CV% and this could be since the distribution of instruments have changed during 2011–2021. Roche and Abbott have all their CVs below 10% in this range, while both Siemens’ groups tend to occasionally go above CV 10% in NT-proBNP levels below 300 ng/L. Nevertheless, when looking at manufacturer-specific median CVs in the 100–500 ng/L range all manufacturers usually perform well below the recommendation of <10%. However, when considering the combined total CV for all manufacturers the CV% rises and more results go above CV 10% at different levels, particularly in samples below 200 ng/L.

When looking with particular emphasis on the last three years 2019–2021 it is evident that the large central laboratory Siemens instruments in the ICA group give substantially higher readings compared to Siemens Stratus, but also to Abbott and Roche. During these three years, Siemens ICA yielded results 16–20% above the consensus median depending on sample level. Similarly, Abbott was 5–7% above, while Roche and Siemens Stratus were 1% and 6–10% below the consensus median, respectively.

The cut-off values do not take the differences of the NT-proBNP methods into consideration, although it would be advisable if such data could be included in the guidelines for HF management, e.g. the situation for troponin where the use manufacturer-specific medical decision limits are well established [29]. The combination of internationally accepted decision values and the bias between manufacturers highlights the need for a common standardisation with higher order reference materials and procedures for measuring NT-proBNP. Particularly since medical decision limits in guidelines, e.g. ESC 2021 [9], which are mostly based on Roche methods, do not take these method differences into account. Accordingly, results from laboratories that use the Siemens ICA might lead to an increase in unnecessary HF investigations with echocardiography. On the other hand, Siemens Stratus tends to yield lower median results, which might lead to a missed diagnosis of HF.

Systematic differences among NT-proBNP levels when measured by different methods show the need for standardisation. These differences may pose problems related to the indication of a reliable clinical cut-off value, which currently is equal for all methods but should be method-specific for HF. Without a proper standardisation considerable effort will be needed to develop method-specific cut-off values for ruling out HF. Thus, there is a need for clinical studies to evaluate the possible differences in clinical accuracy among different NT-proBNP methods when only a cut-off value is used in clinical practice for ruling out patients with suspected HF. The scope could be further extended by investigating how the clinical accuracy is affected regarding the use of NT-proBNP for HF-related diagnostic, prognostic, and monitoring purposes.

This study contains almost 10 years of data from a lot of participating laboratories which covers the most common methods of measuring NT-proBNP in Sweden. A limitation is that there might be other, more popular methods of measuring NT-proBNP in other parts of the world. The samples come from patients, but have been pooled together and frozen, which differs from how samples are handled in a clinical setting. There is also no ‘true’ NT-proBNP value for each sample and thus consensus median has been used instead.

In summary, when looking at each manufacturer separately the results show that NT-proBNP can be measured reliably in the range of the crucial cut-off values of with CVs usually well under the IFCC goal of <10%. However, bias is a major problem, and different manufacturers consistently show readings at different levels. This highlights the need for a common calibrator and reference materials. Thus, further standardisation efforts are needed.