The utility of measurement uncertainty in medical laboratories

Introduction

In the International Vocabulary of Metrology, the measurement uncertainty (MU) is defined as a “parameter characterizing the dispersion of the quantity values being attributed to a measurand” [1]. It describes the interval within which the value of the measurand is assumed to lie with a stated level of confidence. If in the general use, the term “uncertainty” relates to the concept of doubt, in medical laboratory the knowledge of MU implies instead increased confidence in the validity of a measurement result [2], [3].

The knowledge of MU and the definition of its allowable limits for the clinical application of measurements represents one of the mainstays, together with the definition of reference measurement systems and the establishment of a proper post-market surveillance of in vitro diagnostics (IVD) quality, needed to produce standardized laboratory results suitable for clinical use [3]. Estimating and checking MU in medical laboratories is essential to understand its influence on measurement results and, ultimately, their clinical suitability. However, although all medical laboratories seeking ISO 15189 accreditation know that MU estimate is a specific requirement (clause 5.5.1.4), few know what to do with the calculated MU [4].

How to calculate MU in medical laboratories

Historically, two approaches have been proposed for estimating MU: the so-called “bottom-up” and “top-down” approaches [5]. The “bottom-up” approach is the model originally proposed by the “"Guide to the Expression of Uncertainty of Measurement” (GUM) [6]. This model, usually employed by reference laboratories to obtain accreditation according to ISO 17025 and 15195 standards, is based on a comprehensive dissection of the measurement, in which each potential source of uncertainty is identified, quantified and then combined to generate the MU of the result using statistical propagation rules [2]. We previously described the application of this approach to the enzyme measurements using IFCC reference measurement procedures [7]. The application of this approach in medical laboratories is however too complicated and has encountered many practical problems and objections [8].

The “top-down” approach is simpler and represents a good alternative to the previous approach. It estimates MU of laboratory results by using internal quality control (IQC) data to derive the random components of uncertainty and commercial calibrator information. It is now officially endorsed by the ISO Technical Specification 20914 that provides a practical guidance to be applied in medical laboratory settings for the purpose of estimating MU of values produced by measurement procedures intended to measure biological measurands [9]. The inspiring concept behind this approach, described in Figure 1, relies on the definition of MU across the entire traceability chain, starting with the uncertainty of reference materials (u_ref), extending through the IVD manufacturers and their processes for assignment of calibrator values and uncertainty (u_cal) and ending with the random variability of measuring systems (u_Rw)^[1] [10]. In particular, u_Rw can be derived from IQC, while u_cal must include all uncertainties introduced by the selected calibration hierarchy for the measurand beginning with the highest available reference down to the assigned value of the calibrator for the commercial IVD medical device, including the uncertainty of bias correction (u_bias), if a not negligible bias has been detected and corrected by the manufacturer when implementing traceability.

Figure 1:

Sources of measurement uncertainty across the entire metrological traceability chain.

Although reference material providers, IVD manufacturers and medical laboratories have different roles and independent tasks across the metrological traceability chain, their performances contribute together to the MU of patient results [11]. The crucial point is that the estimated MU must be always combined at each level of the employed traceability chain. Particularly, the MU at the level of clinical samples (u_result) must be the combination of all uncertainty contributions accumulated across the entire traceability chain ( u result = √ [ u cal 2 +     u Rw 2 ] , where u cal = √ [ u ref 2 +     u value   assignment 2 +     u bias 2 ,       i f   a n y ] ). This refutes the common misconception that the simple reproducibility of a measurement result equals its overall MU. A correct estimate of MU of laboratory results is indeed not possible without u_cal. In the European market, the information about u_cal shall be provided on request to the professional end-users. Sometimes, calibrators are offered without uncertainty, but it is up to the laboratory professionals to pretend this information and, in case of unavailability, the corresponding material should be disregarded and replaced with some alternatives offering this information needed for the correct estimate of u_result. On the other hand, it is very important to define conditions for deriving u_Rw that should correspond to a within-laboratory reproducibility for a period (e.g. 6 consecutive months) sufficient to capture most changes to measuring conditions and systematic sources of uncertainty, such as those caused by different lots of reagents, different calibrations or different environmental conditions [9]. Characteristics of control material for estimating u_Rw have been defined and should be carefully considered, i.e. the material should be different from that used to check the correct alignment of the measuring system, be commutable and with concentration(s) corresponding to the decision cut-point(s) employed in the medical application of the test [12].

We previously reported some practical examples on how medical laboratories can correctly calculate u_result. As an example, figure 1 of ref. [13] described the metrological traceability chain and combined standard MU of Abbott Architect creatinine enzymatic assay. In particular, u_Rw was estimated as CV from 6-month consecutive measurement data of a serum-based fresh-frozen control material, randomly analyzed daily during our ordinary laboratory activity.

How to define maximum allowable MU

The ISO Technical Specification 20914 is also clear in pointing out that the magnitude of MU should be suitable for a result to be used in a medical decision: “for a given measuring system, estimating the uncertainty of the results produced is of very limited value unless it can be compared with the allowable MU based on the quality of results required for medical use” [9]. After the 1st European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Strategic Conference, held in 2014, objective criteria for defining analytical performance specifications (APS) became available. These criteria are based on three models: model 1, based on the effect of analytical performance on clinical outcomes; model 2, based on components of biological variation of the measurand; and model 3, based on state of the art of the measurement (defined as the highest level of analytical performance technically achievable) [14]. One revolutionary aspect of this approach was to emphasize that certain models are better suited for some measurands than for others, and the attention should therefore primarily direct toward the measurand and its biological and clinical characteristics [15]. Grading different levels of quality (i.e. minimum, desirable and optimum) for APS is also very important because it stimulates IVD manufacturers to work for improving the quality of assays to move, in case, from unacceptable or minimum to desirable performance [16].

For MU, the relevant goal that should be fulfilled is that related to the allowable random variability of patient results, as the correct trueness transfer along the metrological traceability chain should allow the achievement of unbiased (or negligibly biased) results [17]. In a recent paper, we used serum creatinine as an example for the definition of maximum allowable MU [18]. This measurand has a strict metabolic control so that the most appropriate model for deriving APS is that based on its biological variation. By using published data about the average intra-individual biological variation (CV_I) of serum creatinine (4.4%) [19] and the classical Fraser’s paradigm for deriving APS for random variability [20], APS for standard MU of serum creatinine measurement on clinical samples are 3.3% (≤0.75 CV_I, minimum quality), 2.2% (≤0.50 CV_I, desirable quality) and 1.1% (≤0.25 CV_I, optimum quality).

Once defined, APS cover the total MU budget (TB_u) that should be fulfilled at the level of patient results. The achievement of TB_u depends on the MU contributions of each step of the metrological traceability chain [11], [12]. Therefore, it is essential to accurately define the entity of all those contributions and how much of the TB_u is used across the different steps of traceability chain. We previously recommended that specific MU limits at different levels of the traceability chain should be defined as fractions of allowed TB_u; in particular, we conventionally recommended that no more than one third of TB_u should be consumed by u_ref and ≤50% of TB_u used by u_cal. The remaining MU should be available for u_Rw as a margin to fulfill TB_u [11], [12].

The u_ref represents the first contribution to the TB_u. Due to uncertainty propagation in the calibration hierarchy, u_ref may significantly affect the MU of patient results. It is therefore intuitive that it should be markedly lower than APS for TB_u. Continuing to use serum creatinine as an example, Figure 2 reports how to derive the allowable limits for the standard MU of higher-order references in order to not exceed with a high probability TB_u at the level of clinical samples [18].

Figure 2:

Allowable limits for the standard measurement uncertainty (MU) of serum creatinine results on clinical samples (derived from the biological variation model) (u_result) and corresponding limits for creatinine higher-order references (materials or procedures) (u_ref) and commercial calibrator (u_cal), expressed as a fraction of the total uncertainty budget (TB_u).

The role of IVD manufacturers is to identify higher-order metrological references and, based on them, to define a calibration hierarchy to assign traceable values to system calibrators and estimate their MU [21]. The basic paradigm here is that, if present in a not negligible amount, a systematic error (bias) should be appropriately eliminated by adjusting the value assigned to the calibrator, while the overall MU increases because of the u_bias contribution [22]. In addition to u_bias, the manufacturer must also combine u_ref. Once estimated, u_cal should be compared with MU limits, which represent a proportion, e.g. 50%, of the TB_u allowed for clinical laboratory results (Figure 2). Keeping u_cal to a level fulfilling clinical needs is however a highly disregarded issue [13], [23], [24], [25]. When higher-order references do not exist, commercial calibrators are usually value-assigned by manufacturers using in-house procedures. However, even in this case, end-user calibrator assigned values will have an MU that contributes to the u_result. In these circumstances, u_cal will simply correspond to u_{value assignment} (see description in the section “How to calculate MU in medical laboratories”), with no contribution from u_ref and u_bias.

In a previous paper, we provided simulations about the status of the uncertainty budget for some measurands [11]. The indicated approach should now be applied to each analyte measured in the medical laboratory to verify if the status of the uncertainty budget of its measurement associated with the selected metrological traceability chain is suitable for clinical application of the test.

How to deal with bias on clinical measurements

In the traceability framework, medical laboratories should rely on the IVD manufacturers who must ensure traceability of their measuring systems to the highest available references. Therefore, regular estimation of bias by the end-user laboratory is not required. As the IVD measuring system is CE (“Communautés Européennes”)-marked and correct alignment to higher-order references is expected, laboratories should just consider the MU of the value assigned to the calibrator (that should include the u_bias, if any) and combine it with u_Rw to obtain u_result. Appearance of a medically unacceptable measurement bias could be however shown by external quality assessment (EQA) surveillance, but caution needs to be exercised as only schemes fulfilling category I/IIA criteria are usable to this scope [13], [26]. If a medically significant bias is suspected during ongoing EQA surveillance, the bias against a reference (material or procedure) for that measurand should be estimated and the presence of a significant systematic error confirmed. Then, the bias value should be included in the estimate of MU of clinical samples [23], [27]. If the recalculated MU is not fulfilling the predefined APS, it is the responsibility of the manufacturer to take an immediate investigation and eventually fix the problem with a corrective action (e.g. by improving the calibrator value-assignment protocol). If unsolved by the manufacturer, the laboratory could introduce a correction factor for the detected bias. If so, the uncertainty of the correction factor needs to be estimated and included in the calculation of u_result. The use of bias correction factors by individual laboratories is however not permitted by some national regulations as this may alter the status of the measuring system, removing any responsibility from the manufacturer and depriving the system (and, consequently, the produced results) of the certification originally provided through CE marking. The introduction of correction factors by individual laboratories is also quite risky, as they are usually unaware of possible subsequent changes made by the manufacturer and may continue to use the correction factor even when the bias has been corrected in the reagent production stage. Therefore, we are not supporting the individual use of bias correction factors in daily practice, but we strongly believe that involved laboratories should insist in order that the providing manufacturer quickly solves the issue. Table 1 summarizes the suggested sequential approach in case a clinically significant bias on patient results is suspected.

Why MU matters in medical laboratories

Conclusions

In this paper, we provided an overview on how MU should be correctly estimated by medical laboratories and about the importance of this activity to guarantee reliability and quality of provided results. These concepts should apply to both marketed measuring systems and in-house procedures, when CE-marked commercial alternatives are not available. Medical laboratories should estimate (by using the “top-down” approach) and validate (by suitable APS) the MU of each test at the level of patient results. This activity is useful for several reasons, summarized in Table 2, all aimed at improving the quality of patient results provided by medical laboratories and, ultimately, the patient safety [33]. Considering that MU should be used to evaluate both the performance of an IVD measuring system and the laboratory itself, it should be added to the list of key quality indicators in all laboratories [34]. If needed, all attempts must be made to improve critical situations and, if the MU cannot be sufficiently reduced in order to fulfill APS, a decision can be made as to whether the measurement procedure is to be replaced with another performing better.