Percentile transformation and recalibration functions allow harmonization of thyroid-stimulating hormone (TSH) immunoassay results

Introduction

According to all international guidelines, measurement of serum thyroid-stimulating hormone (TSH) is considered the first-line screening test for thyroid dysfunction throughout all the lifespan (including pregnancy, postpartum and neonatal periods), for the evaluation of thyroid hormone replacement in patients with primary hypothyroidism, and for the assessment of suppressive therapy in patients with follicular cell-derived thyroid cancer [1], [2], [3]. Currently, levels of TSH are routinely assayed with non-competitive immunoassay methods [1]. Although the analytical performance of the TSH immunoassays has progressively improved in the last 30 years [1], there are still some systematic differences between the commercially available methods [4], [5]. Systematic bias or difference in interferences between TSH immunoassays may produce misleading interpretation when samples of the same patients are measured by different methods, especially in patients treated with drugs which are able to affect thyroid function or to cause interferences in immunoassay systems [6], [7], [8], [9].

In 2010 [10], [11], [12] and 2014 [13], the IFCC Working Group for Standardization of Thyroid Function Tests published several studies concerning the standardization of both TSH and thyroid hormone immunoassay methods. More recently, the same group reported the results of another study on the evaluation and harmonization, rather than standardization, of TSH immunoassay methods, commercially available at that moment [14]. The aim of the study was to promote harmonization of immunoassay methods for TSH, because a process of standardization was not considered possible owing to the lack of accepted reference measurement procedures (RPMs) for this hormone [14]. Indeed, consensus has been progressively accumulated in the last few years about the importance of a more global approach to achieve a better harmonization in laboratory medicine, especially in the field of the most popular immunoassay methods [15], [16], [17], [18], [19], [20], [21], [22].

A multicenter study, performed on behalf of the Italian Section of the European Ligand Assay Society (ELAS), evaluated the systematic differences between the most popular TSH immunoassays in Italy [5]. The conclusions of this study were that it is possible to produce a mathematical approach, which can obtain a better harmonization between the different TSH methods [5]. After recalibration, performed using a mathematical approach based on the principal component analysis, the variation of TSH values significantly decreased from a median pre-calibration value of 13.53% (10.79%–16.53%) to 9.63% (6.90%–13.21%).

Another important clinical problem related to the routine use of TSH assays is the large between-method difference in reference interval (RI) values [1]. The upper limits of the reference population of TSH immunoassays are strongly affected by outlier values, related to individuals with thyroid autoimmunity (thyroid autoantibody positive) or sub-clinical thyroid disease. Other factors related to population demographics (such as age, sex, and ethnicity), iodine intake, BMI, smoking status and administration of some drugs can affect the serum TSH levels and generate false-positive TSH elevations as well [1], [6], [7]. Further, analytical interferences such as heterophile antibodies have been reported to affect the reproducibility of results from healthy population [23]. For these reasons, the accurate evaluation of TSH reference values requires the enrollment of a very large number of rigorously screened normal euthyroid volunteers [1], [24]. Considering these inclusion and exclusion criteria for the enrollment of the reference population, as well as the systemic bias between TSH immunoassay methods, it is not surprising that RIs for TSH have remained poorly defined [2], [13].

For the calculation of RIs for TSH, some recent studies have suggested some experimental approaches based on indirect RI calculation, using very large populations with the aim of reducing these drawbacks [25], [26], [27], [28], [29]. However, robust statistical analyses, including resampling approaches, are needed for the accurate exclusion of all possible outliers [30], [31]. Theoretically, data mining approaches may be less accurate in the calculation of TSH reference values than the experimental studies based on large reference population. As a result, reference intervals, calculated with data-mining techniques, usually require an independent and accurate evaluation of their clinical effectiveness and efficiency using specific clinical studies.

In 2017, the Italian section of the European Ligand Assay Society (ELAS) organized a multi-center study (named ELAS TSH Italian Study) among several Italian clinical laboratories for the evaluation of TSH RIs using large laboratory databases. Preliminary results obtained from four Italian clinical laboratories, using the same method for the measurement of serum TSH confirmed that data-mining techniques can be used to calculate clinically useful RIs for TSH [29]. Prompted by these preliminary results, the authors extended the experimental protocol of the TSH Italian Study to include data related to three different TSH immunoassay methods collected by seven Italian clinical laboratories.

The aim of the present study was to develop and validate a new approach for harmonizing TSH results, based on regression equations and bootstrapped statistical methods. For this purpose, the TSH results from a large database of seven clinical laboratories were used [29]. Results were then validated on a different already published database to check the performances of RCALs. Finally, age- and gender-specific RIs of TSH were derived after recalibration.

Materials and methods

Results

Distributions of TSH values

The descriptive statistics of the distributions of TSH values obtained by the LIS of the clinical laboratories of seven Italian city hospitals is reported in Table 1. After log₁₀ transformation, the overall data showed a deviation from linearity in the upper and lower boundaries (Supplementary Figure 1). Exact permutation tests showed statistical significances between Access and Architect (p=0.0002, 95% CI: 0.0001 and 0.0007), between Access and Architect (p=0.0002, 95% CI: 0.0001 and 0.0010) and between Access and Elecsys (p=0.001, 95% CI: 0.0001 and 0.0101). Furthermore, descriptive statistics were used for the bootstrapped distribution that was generated by random sampling with re-substitution of 5000 observations for each method, which were all combined together to obtain a final merged distribution of 15,000 TSH values. Table 1 also reports the claimed manufacturers’ RIs for TSH values (mIU/L) obtained by the following inserts: Access, version B83033 D August 2016; Architect, version G3-3871/R04 2012-09 and Elecsys version 07028091500-2.0 2017-09.

Table 1:

Descriptive statistics of TSH values (mIU/L) measured by the three immunoassay methods tested in this study and of the bootstrapped distribution.

Distribution of TSH values, mIU/L							Manufacturers’ claimed RI
Data	Size, n	Mean	Median	SD	2.5th pct (90% CI)a	97.5th pct (90% CI)a
Access	89,657	1.985	1.705	1.233	0.403 (0.399–0.408)	5.224 (5.166–5.272)	0.45–5.33b
Architect	14,109	1.756	1.528	1.055	0.406 (0.390–0.420)	4.512 (4.420–4.620)	0.35–4.94c
Elecsys	21,653	2.090	1.820	1.266	0.420 (0.410–0.430)	5.460 (5.380–5.560)	0.270–4.20d
Bootstrapped distribution	15,000	1.944	1.674	1.1967	0.411 (0.400–0.420)	5.114 (5.030–5.210)	–

1. Manufacturers’ reference intervals (RIs) were also reported. SD, standard deviation; 95% CI, 95% confidence intervals; SE, standard error estimated by SD of the Monte Carlo Results. ^aEstimated by non-parametric bootstrap method, with M=1000 replicates; ^bestimated by using approximately 400 subjects of a general population of approximately equal numbers of males and non-pregnant females between the ages of 21–88; ^cestimated by using 549 reference subjects with normal free T4; ^destimated by 2.5th and 97.5th percentiles of 516 reference subjects.

Figure 1:

Coefficient of variation in percentage (CV, %) of EQA sample results obtained from three different immunoassays, plotted against the mean TSH value (mIU/L).

Dotted line shows the median CVs % (10.72% before and 8.61% after RCAL).

Estimation of TSH reference intervals after RCAL, according to age classes and gender

A published dataset, made of a series of n=142,821 TSH values stored in the LIS of four Italian city hospitals’ clinical laboratories, was used to estimate TSH RI after RCAL. All laboratories used the same analytical method for TSH determination, the Access. Original data (before RCAL) showed a median TSH value equal to 1.75, with 2.5th and 97.5th percentiles being 0.36 and 5.28, respectively [29]. TSH distributions, before and after RCAL, resulted in statistically significant differences (Wilcoxon rank sum test, W=1×10 e¹⁰, p<0.0001), being the 97.5th percentiles 5.28 mIU/L and 5.11 mIU/L, respectively. Considering gender, statistically significant differences were found between the female and male TSH values after RCAL (X²=465.01, p<0.001) (Figure 2). Thus, the following age classes were considered: (a) age <35 years, (b) 35 ≤age <50 years, (c) 50≤age<70 years and (d) age ≥70 years. TSH values after RCAL were statistically significant different among age classes (X²=1038.58, Bonferroni’s adjusted p-value <0.001 in all comparisons), even after applying a further stratification for both females (X²=439.34, Bonferroni’s adjusted p-value <0.001 for all comparisons) and males (X²=722.23, Bonferroni’s adjusted p-value <0.001 for all comparisons). Table 5 reports the median, IQR, RI 2.5th and 97.5th percentiles (and the corresponding 90% CI) of RCAL TSH values, overall, stratified by gender or by gender and age classes. Comparison of percentiles showed that in the female group, 2.5th and the 97.5th percentiles differ across age classes, except for the comparison of the 2.5th percentile between groups (a) vs. (c) (pairwise permutation tests across groups for percentiles, p-adjusted=0.287). In the male groups, significant differences were found across age classes for the 97.5th percentile, while in the case of 2.5th percentiles, statistically significant differences were found for the comparisons between groups (a) vs. (d), (b) vs. (d) and (c) vs. (d).

Figure 2:

Scatterplot of log₁₀ TSH values (obtained after RCAL) and age.

Cubic spline was used to obtain the TSH trend for female (red) and male (blue) subjects.

Table 5:

Descriptive statistics, 2.5th and 97.5th percentiles (and the corresponding 90% CI) of data distribution of TSH values (mIU/L) measured by different laboratories with the method Access after recalibration equations (RCALs).

Access	Median (IRQ)	2.5th (90% CI)	97.5th (90% CI)
After RCAL
Overall data	1.72 (1.14–2.54)	0.37 (0.36–0.37)	5.11 (5.06–5.13)
Female	1.77 (1.16–2.60)	0.35 (0.34–0.36)	5.16 (5.29–5.39)
Male	1.62 (1.09–2.37)	0.40 (0.39–0.41)	4.92 (4.83–4.98)
Females
Age <35	1.87 (1.31–2.64)	0.49 (0.46–0.51)	4.96 (4.87–5.05)
Age ≥35–50	1.77 (1.21–2.53)	0.38 (0.37–0.40)	4.81 (4.73–4.92)
Age ≥50–70	1.76 (1.14–2.61)	0.33 (0.32–0.34)	5.13 (5.07–5.21)
Age ≥70	1.70 (1.05–2.63)	0.31 (0.29–0.32)	5.48 (5.41–5.54)
Males
Age <35	1.96 (1.41–4.88)	0.66 (0.64–0.70)	4.88 (4.74–5.05)
Age ≥35–50	1.62 (1.14–2.30)	0.49 (0.47–0.51)	4.65 (4.52–4.73)
Age ≥50–70	1.55 (0.39–2.27)	0.39 (0.37–0.41)	4.73 (4.63–4.83)
Age ≥70	1.54 (0.99–5.27)	0.35 (0.33–0.36)	5.27 (5.18–5.36)

1. Data considered overall or subdivided by females and males, with or without stratification for age-classes, are reported. Bootstrap statistics (with R=1000 iterations) was used to derive 90%CI of the 2.5th and 97.5th percentiles.

Discussion

Thyroid dysfunction represents an important endocrine disorder, with a prevalence of 3.82% of cases in Europe [34] and in the clinical setting, TSH laboratory testing is considered key to attaining quality medical care in patients with suspected disease. Indeed, TSH has been recently included in the list of the Top 25 Laboratory Tests by Volume and Revenue [35].

In this study, we investigated and validated a novel statistical approach for harmonizing TSH results, using a workflow based on a two-phase approach. In the first phase, RCALs were estimated using TSH values of a huge explorative dataset obtained from LIS of seven different laboratories of Italy, and a validation phase, performed using two different testing datasets.

The basic assumption of the approach proposed in this study is that TSH values obtained from different immunoassays represent subsamples of the whole distribution of TSH, the population data. Considering that each method measures subsamples originating from the same distribution, harmonization can be obtained by recalibrating through merged data (randomly resampled) from different immunoassays. The percentile transformation implemented in this study is similar to the “percentile normalization” approach, used for normalizing other types of data (e.g. microarray data), a technique for making two distributions similar for statistical properties. Interestingly, this is a distributional “constrains-free” approach and does not suffer from the limitations that are typical of parametric methods, such as deviation from normality and outliers. To our knowledge, the usage of bootstrapping resampling and of percentile transformation represents a novel approach for harmonizing immunoassay distributions of TSH values.

In the first phase, the dataset used included a series of 125,419 TSH values and after outlier removal, average and median values of the three distributions were different. For Access and Architect, the estimated RIs were wider than the manufacturers’ declared values, while for Elecsys they were closer than RIs declared into the insert. However, TSH results being highly skewed, data were log₁₀ transformed before further analyses. Shapes of method distributions and of the overall distribution (made from all method results) deviated from the normal, especially for the large left and thick right tails, which might underline that subjects with normal TSH, but with values at the boundary or exceeding the range 2.5th and 95th percentiles, are relevantly under- or overestimated by the different methods, especially for the Architect results (Supplementary Figure 1). Percentile transformations were applied to transform data. As stated above, the applicability of this method is not limited by deviation from the log-normal TSH distributions.

RCAL results are obtained by implementing the robust regression (Huber method) with bootstrap resampling. Regression intercepts and slopes, calculated by averaging the bootstrapped results of several iterations, presented different meanings: the first parameter is used to correct for a systematic component across methods; in contrast, slopes account for random, method-dependent, variations. Confidence intervals obtained by bootstrap statistics showed that for Access, intercept was not statistically significant, while for Elecsys, slope was not significant.

RCAL equations were validated by using two other datasets. Firstly, the EQA results were used to assess the validity of RCAL equations. The comparison of EQA distributions of TSH values before and after applying RCALs on log₁₀-transformed values showed a good agreement among results, the median, 25th and 75th percentiles values being more similar (Table 3 and Supplementary Figure 2). Also, the variability of the results obtainable by measuring the same EQA samples with the three methods are more comparable, being the CV values highly significantly reduced after RCAL (Figure 1).

A second dataset, containing numerous TSH results obtained by simultaneous measurements using the Access, Architect and Elecsys methods, was further used to verify the harmonization properties of RCAL equations. Each method was recalibrated against the other two, in order to achieve a direct method-to-method comparison. The results reported in Table 4 showed that, after recalibration, agreements were very good across all methods, the median of differences ranging from −0.0053 to 0.1955 mIU/L of TSH. Statistical testing of paired data showed that results from Elecsys recalibrated to Access and vice-versa belong to the same data distribution. These results demonstrated that distributions of TSH results for different immunoassay methods can be harmonized to originating comparable results.

A further huge dataset, derived from TSH stored in the LIS of four Italian city hospitals’ clinical laboratories, was used to estimate RIs of TSH after RCAL. Results showed that before and after RCAL results lead to different RIs, especially considering the upper limit and that TSH values are age and gender correlated (Figure 2). Therefore, age- and gender-specific RIs were calculated (Table 5). Especially for the lower RI limit, statistically significant differences were observed stratifying the TSH values for age and gender. Interestingly, while the upper bound of age-specific RIs showed a “U-shape” trend, the lower bound decreased with increasing age. These results shows a decreasing trend of TSH with age, and are in accordance with previously published results of the same group [29], while in the study by Lo Sasso et al., made in the same geographical area and with a different analytical method, the “U-shape” was less remarked in the upper RI limit. In the latter study, RIs were 0.18 mIU/L (90% CI 0.14–0.21) and 3.54 mIU/L (90% CI 3.18–3.90), underlining a marked difference, especially in the upper RI limit [36].

Previous studies from Clerico et al. [4] and Stöckl et al. [14] used other statistical approaches for harmonizing TSH results, reporting successful results. Both studies used robust factor analyses and are based on TSH results obtained from the measurement of a relatively low number of clinical and/or quality-control samples with some TSH methods. In contrast, in the present study, huge datasets were used both to derive RCALs and to validate their efficacy in harmonizing TSH distributions. Two-phase approaches allow reducing “overoptimistic” results that can be obtained sometimes when single-step methods are used. Further, the usage of the large dataset increases the generalizability of RCAL estimations, increasing the applicability of the obtained results [37]. Other advantages with respect to previous TSH harmonization attempts [4], [14] are represented by the combination of percentile transformation with robust regression for estimating the RCAL, and the usage of bootstrapping for iterating the whole procedure. On the one hand this allow to reduce the influence of large left and thick right tails on RCAL estimations. On the other hand, during bootstrapping, subsamples of the same size are considered and merged several times, further limiting the effects of outlier TSH results [30].

The aspect of accounting interferences is of utmost importance for the harmonization of TSH results [8], [9]. Potential interferences in immunometric methods with rheumatoid factors or heterophilic antibodies can occasionally cause abnormal TSH concentrations [38], [39]. Further, samples with elevated levels of biotin or anti-streptavidin antibodies might produce variability between analyzers [38], [39]. Interferences in immunometric methods could be considered as outliers [8], [9], and their frequency for TSH testing is estimated to be up to 1% [38]. Accordingly, the robust statistical approach used in this study should also be able to eliminate the interferences, which produce TSH values above or below the reference range (i.e. abnormal values).

The major disadvantage of the approach proposed in this study with respect to that proposed by Clerico et al. [4] and Stöckl et al. [14] is the high complexity of statistics due to the bootstrapping application, even if the usage of these resampling methods is limited to the initial setup of RCAL.

In conclusion, the present study represents insights on the utilization of the percentile transformation approach and of bootstrapping statistics to derive RCALs, which are useful to harmonize TSH values. Further, this approach can lead to the estimation of method-independent RIs for TSH. On the other hand, this approach also presents some important limitations. For example, robust and valuable RCAL estimations require the availability of big data sources, collected possibly from different laboratories and in different time periods, which complicated the applicability of bootstrapping methods that are computationally intensive.