Extensive analytical evaluation of the performances of the new DiaSys PCT assay and comparison with Elecsys B·R·A·H·M·S PCT test on Roche Cobas and B·R·A·H·M·S PCT-sensitive Kryptor

To the Editor,

In the last years, procalcitonin (PCT) has been extensively evaluated and applied in different clinical settings [1], [2], [3], particularly for the diagnosis of sepsis, severe sepsis or septic shock [1, 4] and, more recently, for lower respiratory tract infection (LRTI) and antimicrobial stewardship [4]. Different tailored cut-offs and algorithms have been defined using the B·R·A·H·M·S PCT assay. In this work we evaluated the analytical performances of the new particle enhanced immunoturbidimetric DiaSys PCT test (PCT-DiaSys) and compared it with two automatic assays, the Elecsys B·R·A·H·M·S PCT test (PCT-Elecsys) and the B·R·A·H·M·S PCT-sensitive Kryptor test (PCT-Kryptor). The PCT-DiaSys (DiaSys Diagnostic Systems GmbH, Holzheim, Germany) and the PCT-Elecsys (Roche Diagnostics, Italy) assays were installed, respectively on the module c702 and on the module e801 of the Roche Cobas 8,000 analyser (Roche Diagnostics, Monza, Italy), while the PCT-Kryptor assay was installed on the Kryptor Compact Plus instrument (Thermo Fisher Scientific, B·R·A·H·M·S GmbH, Germany). The PCT-Elecsys and PCT-Kryptor methods are routinely used in the laboratory and validated in a previous study [5]. In this study, the PCT-DiaSys method was extensively evaluated by CLSI guidelines, in particular CLSI EP05-A3 for imprecision [study design: 80 measurements/concentration (2 replicates, 2 runs/day, 20 non-consecutive days) at 6 concentrations (4 serum pools and 2 control materials)], CLSI EP17-A2 for Limit of Quantitation (LOQ) [study design: 100 replicates (precision profile based on 10 pools closed to LOQ declared by the Manufacturer, run as duplicates during 5 days)], CLSI EP06-A for linearity [study design: 2 curves (1: range 0–30 μg/L; 2: extended range 0–150 μg/L), 11 levels/curve, read as triplicates in a single run (33 replicates/curve); goal of 5 % for deviation from linearity)]. Methods were compared by measuring, across 10 non-consecutive days, 152 samples selected from the clinical laboratory routine and stored at −80 °C [sample distribution: <0.25 μg/L: n=42 (28 %), 0.25–0.5 μg/L: n=35 (23 %), 0.5–1.0 μg/L: n=21 (14 %); 1.0–2.0 μg/L: n=13 (9 %); 2.0–10.0 μg/L: n=25 (16 %), >10 μg/L: n=16 (10 %)]. Agreement between methods was also evaluated by a new graph, called “Moving Cohen’s kappa graph”, presented here for the first time, where Cohen’s kappa values, calculated for each method, are plotted against cut-off values.

Within-lab or total laboratory variability of PCT-DiaSys ranged between 6.29 and 12.48 %, higher than those reported by previous works [6, 7]. The total variability of the PCT-Elecsys test, routinely used in our laboratory, is <4 %, as estimated in the long period (6 months) by QC materials. This is in accordance with data shown in previous works, where for Elecsys B·R·A·H·M·S PCT, at concentrations of PCT ranging from 0.49 to 10.09 μg/L, a total imprecision between 2.96 and 1.70 % was reported [5]. LOQ (0.18 μg/L at CV=10 %) and linearity (up to 80 μg/L) were in accordance with what has been declared by the Manufacturer. Given higher imprecision and lower sensitivity intrinsic to the PETIA technology, the lower analytical performances observed for PCT-DiaSys were expected.

The results of Passing–Bablok regression and Bland–Altman analysis for method comparison study are reported in Table 1. A statistically significant constant systematic error, but no proportional error, was found when PCT-DiaSys was compared with PCT-Kryptor (intercept: 0.085, 95 % CI 0.058–0.110); when compared with PCT-Elecsys, both a constant and a proportional error were evident (intercept: 0.078, 95 % CI 0.042–0.104; slope: 1.205, 95 % CI 1.148–1.254), with higher PCT values obtained by PCT-DiaSys, and an overall bias between PCT-Elecsys and PCT-DiaSys of −0.97 μg/L or −41.6 %. The bias between PCT-Kryptor and PCT-DiaSys was lower and closer to the bias observed for the comparison PCT-Kryptor vs. PCT-Elecsys (Table 1). Slope and intercept estimated in this study for PCT-Kryptor vs. PCT-Elecsys are very similar to those reported in the IFU (Table 1).

Concordance between methods was also evaluated at different clinical cut-offs, validated for sepsis/systemic infection (0.50, 1.00 and 2.00 μg/L), as percentage of concordant pairs and by Cohen’s kappa (Table 1): the agreement between PCT-DiaSys and PCT-Kryptor or PCT-Elecsys was lower than between PCT-Kryptor and PCT-Elecsys at each clinical cut-off (Table 1).

PCT assays were further compared at levels below 0.50 μg/L, currently used for acute respiratory tract infections. At these levels, a larger scattering around the regression line was appreciable for the comparisons PCT-Kryptor vs. PCT-DiaSys or for PCT-Elecsys vs. PCT-DiaSys (data not shown). Moreover, the concordance at 0.25 μg/L with the PCT-DiaSys was found to be very low, respectively 87.5 % (% concordant pairs) and 0.58 (95 % CI 0.41–0.73) (Cohen’s k) for PCT-Kryptor vs. PCT-DiaSys, 83.6 % and 0.50 (95 % CI 0.33–0.65) for PCT-Elecsys vs. PCT-DiaSys (Table 1). Concordance at 0.25 μg/L for the pair PCT-Kryptor – PCT-Elecsys was substantially higher, being 96.1 % and 0.90 (95 % CI 0.81–0.98). The difference in concordance was more evident when plotted according to the new Moving Cohen’s kappa graph presented here for the first time, where for increasing cut-offs (from 0.2 to 2 μg/L, step=0.05 μg/L) a Cohen’s kappa for each comparison is calculated and displayed (Figure 1C). Below 0.50 μg/L, the agreement with PCT-DiaSys is unsatisfactory, while at higher cut-offs the overall concordance with PCT-Kryptor and PCT-Elecsys, could be considered comparable (Figure 1C, Table 1). The agreement at 0.1 μg/L was not calculated due to the higher LOQ of the DiaSys PCT method (0.18 μg/L). Discrepancies at different cut-offs in the lower range of concentrations (up to 1.00 μg/L) between PCT-DiaSys and PCT-Kryptor or PCT-Elecsys are detailed in Figure 1A and B, where red points represent discordant values. The discrepancy observed between PCT-DiaSys and the other two PCT assays may depend on several reasons, including higher PCT-DiaSys imprecision at low PCT concentrations or lower PCT-DiaSys selectivity, but also it could be due to possible variability originating from the diverse antibodies employed by the three assays (two monoclonal for PCT-Elecsys, one polyclonal and one monoclonal for PCT-Kryptor, and polyclonal for PCT-DiaSys) [8].

Figure 1:

Scatter plot for the PCT assays evaluated in this study. (A) Comparison between B·R·A·H·M·S PCT-sensitive Kryptor (x) and DiaSys PCT (y) at different cut-offs. (B) Comparison between Elecsys B·R·A·H·M·S PCT (x) and DiaSys PCT (y) at different cut-offs. Dashed blue vertical lines indicates cut-offs at 0.10, 0.25, 0.50 μg/L. Black points represent concordant pairs along the diagonal of the graph, while red points represent discordant pairs. (C) Moving Cohen’s kappa graph for the comparisons in study. For increasing cut-off (from 0.2 to 2 μg/L, step=0.05 μg/L) a Cohen’s kappa for each comparison is calculated and plotted. Red: PCT-Kryptor vs. PCT-DiaSys, Green: PCT-Kryptor vs. PCT-Elecsys, Blue: PCT-Elecsys vs. PCT-DiaSys.

It is noteworthy that PCT algorithms, devised to initiate or discontinue antibiotics in acute respiratory tract infections and based on 0.10, 0.25 and 0.50 μg/L cut-offs, were shown in several randomized controlled trials, in “real life” scenarios and systematic reviews to effectively reduce antibiotic exposure and antibiotic-associated adverse effects [9, 10]. The worse performance of PCT-DiaSys at <0.50 μg/L, and the impossibility to detect or accurately quantify PCT at <0.2 μg/L, could actually hinder the application of these PCT-based algorithms. However, it must be stressed, and taken into consideration, that DiaSys’ IFU does not indicate any validation for acute respiratory tract infections, with cut-off below 0.5 μg/L, but only for systemic infection/sepsis.

Interestingly, in three cases a significant clinical (normal vs. markedly pathological) difference was found between the value obtained by PCT-DiaSys and those obtained by PCT-Kryptor and PCT-Elecsys, respectively 2.66 vs. 0.20 or 0.18 μg/L, 7.67 vs. 0.39 or 0.35 μg/L and 18.18 vs. 0.28 or 0.23 μg/L. These discrepancies were higher than what could be expected on the basis of method bias and imprecision; however, final diagnosis of these cases was not available, which hinders full data interpretation.

In conclusion, at PCT concentrations ≥0.50 μg/L the new DiaSys PCT assay showed overall good analytical performances and good agreement with the other two B·R·A·H·M·S methods. At PCT levels <0.50 μg/L, the agreement between PCT-DiaSys and PCT-Kryptor or PCT-Elecsys was poor and imprecision was considerably higher.