Biological variation estimates of thyroid related measurands – meta-analysis of BIVAC compliant studies

Quality and characteristics of thyroid BV studies

The systematic literature review identified only one study, the EuBIVAS, achieving the highest BIVAC grade A [21]; one paper from Mairesse et al. [22] was graded B, while the remainder were classified as C (Supplementary Table 1). The highest number of studies were identified for TSH (n=12) and TSI (n=10). Eight BV subgroups from five studies for TG were identified: four of those were included in meta-analysis, one A, one B and two C studies. There was, however, little observable differences in the data described in these studies and in the four studies not included in the meta-analysis. Meta-analysis was not possible for in anti TPO Ab, anti TG Ab and TBG as only one qualifying paper (paper 40, BIVAC Category C) [23] was found. Caution should therefore be exercised when making use of these BV estimates clinically for these measurands because they do not come from meta-analysis but from a single study.

Many factors may impact on the observed estimates of BV of a measurand including population studied, disease status, technology applied, and timing. The characteristics of the study populations such as age, gender and health status all serve to impact upon observed variation. In addition, analytical method standardization and performance, in terms of imprecision and specificity, might be expected to impact upon the value of a measurand [20]. Clearly technological developments have delivered generational changes in analytical systems deployed, meaning that such performance characteristics of the assays have greatly improved with time. Finally, the rhythmical changes described for thyroid hormones mean that differences in timing of sampling and duration of the study, may result in between study variance of BV estimates. All of these factors have to be considered as part of the review process.

Most of the studies found in the literature search were performed before the year 2000 and consequently the analytical methods (mostly RIA) might not be expected to be fully comparable in terms of analytical specificity with those currently in use (ECLIA, CLIA). Taking TSH as an example, we identified two CLIA and four ECLIA studies in comparison with four RIA. In general, the review shows that similar BV estimates were obtained, regardless of the analytical method used. The mean CV_A obtained when pooling CV_A estimates from all studies was 6.4% (RIA CV_A mean: 9.9%, ECLIA: CV_A mean: 2.1% and CLIA CV_A mean: 13.5%, the latter was derived from only one paper [paper 209; analysis performed in a Siemens Centaur]) [24]. Mean TSH concentrations ranged from 1.34 to 2.5 µUI/mL in healthy people. In general, minimal impact on BV estimates for other measurands was observed attributable to the analytical method type applied. This enables use of historical BV data.

Diurnal and seasonal variations in TSH and thyroid hormone measurands have been documented previously [25], [26], [27], [28]. A meta-analysis by Kuzmenko et al. [27] indicated that seasonal changes are seen in TSH, fT3 and fT4, but not for T4. This could lead to differences in BV estimates depending upon study duration.

Three of 12 the studies of TSH identified for this review had a study duration extending to one year. Two of them (paper 240; RIA was performed in healthy individuals and paper 413; ECLIA in children [29, 30]), reported higher CV_I (29% and 28%, respectively) than studies of shorter duration. This may be consistent with super-imposed seasonal variation being included in the BV estimates. The third study (paper 521, employing ECLIA in a study spanning 13 months) [31] identified a CV_I of 16.6%. However, this study population included diseased individuals (subclinical hypothyroidism). None of these studies assessed for trends in their data (Supplementary Table 1), which is a BIVAC criterion. Similar findings applied to T3, fT3, T4, and fT4 in the papers where these hormones are included. Clarification of these relationships requires more studies covering a complete period of one year, with focus on potential trends over time. There is potential here to analyse big data sets from a laboratory information systems to ascertain impact at a population level and extract estimates of BV [31, 32].

Turning to diurnal variations, Liyanage et al. [26] recorded a within-day variability in TSH and fT4 in Sinhalese adults that requires further characterization. This delivers chronobiological complexity as a confounding factor to be understood and accounted for when deriving and applying BV data. Diurnal variation in TSH variation was also identified in a recently published NHANES III US survey, with the peak occurring during the night and the nadir, which approximates to 50% of the peak value, occurring between 10:00 and 16:00 [22]. The authors of this study [25] proposed that this type of BV is not of great significance in terms of clinical interpretation of results as most samplings take place between 08:00 and 18:00 avoiding the night time peak. It will, however, contribute additional higher estimates of BV if the time of day study subjects are sampled is imprecise at an individual or group level. Sviridonova et al. [28] found that the morning median TSH value in the patients with subclinical hypothyroidism was higher than in the afternoon (5.83 mU/L vs. 3.79 mU/L), and they concluded that, according to the TSH reference interval, hypothyroidism could not be diagnosed in about 50% of the cases in the afternoon. Here again there may be a future role for analysis of big data sets to understand the significance of these complex variation states [32, 33].

Our systematic review indicated that state of health has little impact on the CV_I of the measurands reviewed (Figures 1–4). A cautionary note here is that this observation is based on a smaller number of studies in specific disease states such as subclinical hypothyroidism or well controlled hypothyroidism, and it cannot be assumed that this is true of all diseases [34]. Further studies, applying big data approaches could be useful to obtain BV in populations not easily studied in the traditional way, such as children.

Application of BV data

If a measurand exhibits a low II (<0.6), the utility of population based reference intervals is limited for detection of clinically significant results. In this situation, it is better to use the subject under investigation as their own point of reference [35] and to employ RCV to identify significant change between two consecutive test results. A change greater than the RCV indicates significant change at a given probability warranting further action. When the II is low, a patient may have consecutive test results that fall within the population reference range that exhibit a difference that exceeds the RCV. RCV is generally reported as a percentage. Ideally, the II should be greater than 1.4 for population based reference intervals to be applicable, at II values between 0.6 and 1.4 they are of more limited utility. The lower the value of II, then the higher individuality of the measurand and the lower the utility of population based reference intervals.

As seen in Table 2, the II values indicate high degrees of individuality for all the measurands, with the greater majority having values of less than or about 0.6. This finding indicates the use of RCV or personalized reference intervals is preferable in the interpretation of consecutive test results in an individual [17, 20, 35].

Correct interpretation of significance of change in results is clearly important and RCV is considered a valid concept to enable this. We introduce probability curves as a possible tool under development to aid in the process of interpretation. They enable an objective assessment of the statistical significance of an observed change in a measurand. It is of fundamental importance to understand that valid application of RCV, and the derived probability curves for any measurand, requires consideration of the magnitude and constancy of CVA; RCV being a function of two variables, CV_I and CV_A. If applying RCVs clinically, the user must first ascertain whether the CV_I value used in the formula reflects that of the population to which the RCV is to be applied. Secondly, the CV_A utilised must reflect that of the method used to deliver the result locally, ideally meeting the APS goal for imprecision. The curves, an example of which is shown in Figure 5 are then only generally applicable across a range of measurand values if the CV_A is constant across that range. If the imprecision profile of the analytical system shows significant variance along the length of the calibration curve, then the RCV becomes a measurand/magnitude dependent variable. Modern laboratory information management systems could be programmed to appropriately to account for imprecision changes to deliver RCV in real time and used to flag change along with statistical significance. Transportability of RCVs across systems requires well defined and robust estimates of CV_I and equivalence of APS.

The CV_I estimates of free thyroid hormones (fT3 and fT4) appear lower than the estimates for total hormones (T3, T4) concentrations. Thyroid hormones circulate largely protein bound. Free hormone concentrations reflect an equilibrium between protein bound fractions, tissue metabolism and thyroid production. It is logical to assume that, as the free hormones are the biologically active fraction of the total and exert feedback control over thyroid hormone production, that they may be more tightly controlled via homeostatic mechanisms.

Limited availability of data for the thyroid antibody-related analyses leaves a requirement for more studies or again investigation of big data approaches to deliver robust BV data.

Updated APS based upon meta-analysis derived BV data are shown in Table 2. The meta-analysis derived CV_I deliver in general APS for imprecision (CV) that is lower than the mean CV_A estimate of the studies included in this systematic review, excepting TSH and TSI. The lower estimates of CV_I for both fT4 and fT3 deliver APS which may be considered challenging even for the most recent state of the art analytical methods.

Use of analytical systems of equivalent analytical performance and meeting the APS identified in Table 2, will facilitate transportability of RCV values across geography, time and analytical methods used.

In conclusion, this study has delivered a systematic review of published studies of BV of thyroid disease related measurands. An objective assessment of the quality of the studies published by application of the BiVAC has enabled the collation of data sets of sufficient quality to apply meta-analysis and of delivery well characterized estimates of BV for some, but not all measurands studied and reviewed. For some important analytes there were few studies of measurands required for patient care in the many diverse and complex clinical scenarios requiring laboratory testing for healthcare. The review, and subsequent meta-analysis, of experimentally derived data sets, has enabled derivation of BV data estimates that are suitable for the many applications of the data; those including the setting of APS and determination of significance of change in serial results. It is important that manufacturers consider the significance of the new APS (e.g. fT4 and fT3) as it may challenge the performance requirements of their current devices [36].

However, of the many papers graded only the EuBIVAS study fully complied with BIVAC to be classified as Grade A quality. A potentially high degree of complexity has been identified in the BV of the thyroid biomarkers reviewed. Chronobiological factors, the unavailability of BIVAC grade A studies, and the requirement for BV applicable to the many clinical scenarios across varying time scales delivers challenges. Big data approaches for definition may provide an alternative to the stringent experimental approach to definition of BV data required. The prevalence of thyroid testing in populations delivers large volumes of well-characterized data in clinical laboratory databases to support this approach.