Definition and application of performance specifications for measurement uncertainty of 23 common laboratory tests: linking theory to daily practice

Introduction

The estimation of measurement uncertainty (MU) of laboratory results is requested to obtain accreditation of medical laboratories according to ISO 15189:2012 standard [1]. The ISO Technical Specification 20914:2019 provides a guidance on how to estimate MU using the so-called “top-down” approach by combining all sources of MU present in the selected traceability chain [2]. In particular, the MU at the level of clinical samples (u_result) should be the combination of all uncertainty contributions represented by: 1) MU of higher-order references, 2) MU of in vitro diagnostics medical device (IVD-MD) calibrators, and 3) MU associated with the random variability of commercial measuring systems [3]. ISO 20914:2019 also emphasizes that the magnitude of estimated MU should be suitable for a result to be used in medical decisions. The definition of an allowable MU is therefore essential to ascertain if estimated MU for a given laboratory result may significantly affect its interpretation [4]. The Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM), held in 2014 in Milan, established the criteria for defining analytical performance specifications (APS), based on three models: a) the effect of analytical performance on clinical outcome; b) the biological variation (BV) of the measurand; and c) the state of the art of the measurement (defined as the highest level of analytical performance technically achievable) [5, 6]. In a following paper, the criteria for assigning different measurands to each of the three models were made explicit, considering the clinical use of the measurand and its biological characteristics [7].

In a previous study, we defined APS for MU for a first group of 13 measurands according to the above-mentioned models, also briefly providing a preliminary information about their applicability in daily practice [8]. In the present paper, we expanded this approach by analyzing an additional group of 23 measurands, which measurements are frequently requested in Laboratory Medicine. The aim of our study was: a) to categorize the selected measurands according to the Milan models, b) to define APS for MU (at desirable and minimum quality level) using the available information preliminary checked in terms of robustness, and c) to estimate for each test the u_result and compare it with set forth APS to see if today’s measuring systems are able to meet them.

Evaluated measurands and search for information associated to selected Milan models

We identified a list of 23 measurands among the most requested tests in our healthcare system. We considered (in alphabetical order): albumin (serum and urine), serum alkaline phosphatase (ALP), serum aspartate aminotransferase (AST), serum creatine kinase (CK), serum digoxin, serum γ-glutamyltransferase (GGT), serum high-density lipoprotein cholesterol (HDL), plasma homocysteine, serum immunglobulin A (IgA), G (IgG), and M (IgM), serum lactate dehydrogenase (LDH), serum magnesium, serum pancreatic amylase (P-AMY), blood platelets, red blood cells (RBC), serum total cholesterol, serum prostate-specific antigen (PSA), serum total proteins, serum triglycerides, serum urate, and white blood cells (WBC).

For derivation of corresponding APS for MU, each measurand was allocated to the best suited Milan model on the basis of its clinical and biological characteristics, and corresponding information was searched and retrieved as follows:

1. Outcome-based model (model 1). According to previous discussion [6], [7], [8], six measurands were in principle allocated to this model: albumin (serum and urine), total cholesterol and HDL, triglycerides, and platelets. We searched peer-reviewed literature for outcome studies dealing with their clinical use and evaluating the impact of random analytical variability on clinical outcomes (systematic search strategy shown in Supplementary File 1 in the online Data Supplement). The APS for MU were derived by identifying the random variability of patient results corresponding to the misclassification rate which was considered clinically acceptable. However, for four measurands (serum albumin, HDL, triglycerides, and platelets), no outcome-based data in literature were retrieved. Therefore, to derive APS we temporarily allocated those measurands to the BV model [7].

2. BV-based model (model 2). Except for digoxin (see below), all other measurands were allocated to this model. For these measurands, we first retrieved from the EFLM BV database [9] the publications with the highest rate when evaluated for their compliance to the 14 BV data critical appraisal checklist quality items (BIVAC-QI) [10]. We also searched in literature other studies deriving BV of measurands and evaluated them according to BIVAC-QI and other practical guidances (systematic search strategy shown in Supplementary File 2) [11]. For each measurand, the study with the highest score was identified and used to derive APS for MU by the adaption of the classical formulas for deriving analytical goals for random variability from intra-individual BV (CV_I), i.e., ≤0.50 CV_I for desirable and ≤0.75 CV_I for minimum quality level, respectively [4, 8, 12]. We are aware that selecting the study with the highest score does not exactly correspond to the strategy employed by the EFLM working group, which is conversely based on the meta-analysis of available data (although we are aware that in their approach, the group applied different weights to reflect the quality of the articles included in the meta-analysis). When studies of elevated grade are available, we consider better using estimates from such studies alone more than the meta-analysis results. Indeed, the use of meta-analysis can be criticized if included studies show significant heterogeneity and different qualities, especially if the majority are of C grade according to BIVAC-QI, as is not rarely the case in the EFLM database.

3. Model 1&2. According to the concepts elegantly discussed by Fraser [13], drugs which serum concentrations are monitored in laboratory need a specific approach when deriving APS, based on fundamental pharmacokinetic theory and average elimination half-life of the drug. Although the concentration of drugs does not fluctuate randomly around a homeostatic set point, this approach has a relationship with biological knowledge. On the other hand, therapeutic drug monitoring (TDM) is linked to the patient outcome in defining the levels of drug which are potentially toxic or when the treatment can be ineffective. Accordingly, we considered that directly allocating digoxin (and more in general therapeutically monitored drugs) to Milan models 1 or 2 can be incorrect and a sort of hybrid model between the two models, that we called model 1&2, was tentatively proposed.

Note that in this paper all MU data are reported as standard relative MU (u). They can be expanded (U) at 95.45% level of confidence by multiplying by a coverage factor of 2.

Measurands belonging to model 1

Measurands temporarily belonging to model 2

Measurands belonging to model 2

Measurand belonging to model 1&2

MU of laboratory tests evaluated and compared with established APS

As mentioned in the ‘Introduction’ section, the ‘top-down’ approach for MU estimate, as recommended by the ISO Technical Specification 20914:2019 to medical laboratories, relies on the definition of MU across the entire calibration hierarchy assuming that all the significant systematic error (bias) is estimated and corrected by the IVD-MD manufacturer, and the uncertainty of correction (u_bias) defined [3]. u_result is calculated using the equation √(u_cal² + u_Rw²), where u_cal=√(u_ref² + u_{value assignment}² + u_bias²), if not negligible, and u_Rw is defined by ISO/TS 20914:2019 as “uncertainty component under conditions of within-laboratory precision” [3]. Because more than one metrological traceability option may be available for the transfer trueness process and MU of IVD-MD may be influenced by the selected traceability chain [33], u_cal should be always provided as combined with uncertainties introduced by higher levels of the selected calibration hierarchy. In practice, however, few, if none, manufacturers provide u_cal estimated as described above. What manufacturers are usually providing on request is u_{value assignment} of commercial calibrator, so that the laboratory should independently retrieve, when available, the corresponding u_ref on the basis of higher-order reference declared by the IVD-MD manufacturer and combine it to the former to obtain the correct u_cal estimate.

For the IVD-MDs we tested in this study to compare obtained u_result to the established APS for the 23 selected tests, none of the manufacturers of IVD-MDs provided u_cal combined with the corresponding u_ref, even when, as in the majority of cases, this was available. Manufacturers were asked for metrological traceability information in order to identify the higher-order references (materials and/or procedures) used to assign traceable values to their calibrators and obtain a description of the applied calibration hierarchy. Abbott and Roche issued traceability and MU documents where only the names of higher-order references of each analyte are enlisted, without any indication regarding the internal procedure followed for the implementation of the selected metrological traceability. Beckman Coulter and Sysmex, in providing certification of the calibrator uncertainty (as u_{value assignment}) also attached documents reporting a general description of applied calibration hierarchy. So that, in the majority of cases, we retrieved by ourselves u_ref, when available, from the certificate of analysis of the stated reference material found on the website of material supplier. A different approach was used for enzymes that (except for P-AMY) in the Abbott Alinity c system are calibrated using a calibration factor instead of a calibrator material. In this case, u_result was obtained as previously described in a validation study [34].

Basics to the ‘top-down’ approach for estimating MU is the correct estimate by the medical laboratory of the random variability (u_Rw). u_Rw should be derived from internal quality control (IQC) data as described in detail in a previous paper [35]. In this study, results of daily measurements of control materials obtained over a 6-month period were employed as recommended in the ISO 20914:2019 Technical Specification [2]. As previously described, the IQC material employed for u_Rw estimate should be different from that used for the verification of measuring system alignment, be commutable and, if possible, with measurand concentrations near to the employed clinical cut-points [3, 35, 36]. When commercially unavailable, in-house IQC materials should be prepared (e.g., by arranging pools of selected samples). In addition, IQC material measurements should be performed randomly inside the analytical run, mimicking measuring conditions of clinical samples, under properly verified system alignment [35].

Table 2 reports MU contributions for evaluated measurands when measured with the measuring systems available in our laboratory, highlighting the fulfillment (or not) of the established APS. Our data can be used for directly answering to the question: “how many measurands can achieve the recommended level of MU and can the tested measuring systems hit these targets?”. As shown, 16 tests fulfilled desirable and 5 minimum APS for MU, while only 2 assays (serum albumin and plasma homocysteine) exceeded them. MU associated with a serum albumin result of approximately 40 g/L (3.54%) was about two times higher than the minimum APS derived from BV (≤1.88%). Depending from the biology and strict homeostatic control of serum albumin, the analytical quality required for its measurement is extremely high and MU should be as much as possible small. In previous papers, we discussed in detail the issue of MU of serum albumin and how MU of the currently available reference material (i.e., ERM-DA470k/IFCC) is probably not small enough to guarantee the performance needed in terms of MU for the clinical usefulness of the test [4, 37, 38]. We previously recommended that no more than one third of APS for MU should be consumed by the u_ref, letting the remaining allowable uncertainty available for other MU sources in the lower parts of calibration hierachy, i.e., u_{value assignment} and u_Rw [33, 39]. Serum albumin is therefore representative of a measurand for which it should be a priority to significantly reduce the uncertainty associated to the upper levels of metrological chain. On the other hand, parallel strategies focused on reducing the contribution of the lowest parts of the traceability chain to the total uncertainty budget should also be envisaged.

Plasma homocysteine appeared not far (5.67% vs. 5.27%) from fulfilling the minimum APS for MU. It should however be noted that Abbott does not provide for their assay u_ref making the u_result underestimated. More importantly, given the availability of mass spectrometry-based reference measurement procedures, listed in the database of Joint Committee on Traceability in Laboratory Medicine (JCTLM), manufacturers should use them (and not an internal standard) in order to provide traceability to higher-order references and assure that the bias is appropriately eliminated [40]. u_Rw of the employed measuring system was also quite wide, so that an increase in its precision should be considered by the manufacturer.

Except for these two measurands for which further improvements in the quality of their measurements (at least in the particular tested systems) are needed, in our working conditions the remaining 21 tests fulfilled at least the minimum APS for MU. Further research is certainly needed to see if other commercial measuring systems can achieve or not these APS. However, we should preliminarily consider the proposed APS for MU realistic and not impossible to fulfil. One potential limitation of our study was that only one concentration level for IQC material was employed to obtain u_Rw in order to simplify the protocol of this preliminary evaluation study. The importance of more than one level for the accurate estimate of MU of the test has been previously underlined because MU may vary with the analyte concentration [36].