Critical appraisal and meta-analysis of biological variation estimates for kidney related analytes

Introduction

Reduced kidney function correlates strongly with increased morbidity and mortality, and biochemical measurements have a central role in diagnosing and monitoring of kidney disease and the effectiveness of treatment. Essential for interpretation of laboratory test results in these settings is knowledge of the within-subject biological variation (CV_I) of the analytes. Furthermore, data that characterize the CV_I and between-subject biological variation (CV_G) can be used e.g. for the development of analytical performance specifications [1] for internal quality control [2] and for external quality assurance [3], [4].

Serum creatinine and urea are used to assess patients either at risk for kidney disease, or to monitor those having chronic kidney disease (CKD). When reviewing whether a change in serial measurements is potentially of clinical significance or consistent with normal biological variation (BV), it is important to assess the BV in a setting that is comparable to the clinical setting, thus delivering a valid point of reference to enable safe application of the data. This means that if a patient has their kidney function measured every three months, BV data based on weekly measurements may not be applicable. Thus, for the calculation of reference change values (RCV) [1], appropriate CV_I estimates must be applied. The RCV also takes into account the analytical (CV_A) variation of the measurement method to describe the maximum expected change between two measurements at a predetermined level of probability. Changes in serial results over time exceeding the RCV greater than might be expected given relevant estimates of CV_I and CV_A are likely to represent clinical important changes in renal function.

Many other important laboratory applications of BV data also depend upon the availability of estimates of the components of variance that are well characterized and are of sufficient quality and thus transferable to the population to which the laboratory tests are to be deployed. As a consequence, concerns have been raised around the quality of existing BV studies and the veracity of the published BV data estimates that were collated in the historical BV database, last revised in 2014 [5], [6]. With this background the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Biological Variation and the Task Group on the Biological Database have developed the recently published Biological Variation Data Critical Appraisal Checklist (BIVAC) [7]. The BIVAC enables an objective assessment of the quality of BV publications by verifying whether all essential elements that may impact upon veracity and utility of the derived BV estimates are present [7], [8]. It thus both provides a tool to review historical data and to assess new studies and has been applied here to enable a systematic review process applied to measurands commonly used in renal medicine.

The aims of the present study were:

1. To perform a systematic literature review of BV studies published for the following analytes used in the diagnosis and management of kidney related disorders; albumin, creatinine, cystatin C, chloride, potassium, sodium and urea in serum and plasma,

2. To critically appraise relevant publications on these markers by application of the BIVAC [7].

3. To review the effect of different study population characteristics and sampling intervals on associated BV estimates, and

4. To perform a meta-analysis of BIVAC compliant studies with comparable study design to deliver global BV estimates for kidney related analytes

Materials and methods

The initial source material identified for review was the historical BV database, from which relevant publications on kidney markers were retrieved (references 7–246 in Supplementary Table 1) [5], [6]. Furthermore, multiple systematic literature searches were performed in PubMed, as described in detail in [7] with cut-date Feb 7th, 2020, identifying more than 200 additional publications on BV, 26 of which addressing kidney markers (references 248–483 in Supplementary Table 1). The retrieved publications are identified in this review by the article number they have been assigned in the EFLM BV database [9] (Supplementary. Table 1). When one study included estimates for several different subpopulations, such as for instance based on sex, age or health status, a subscript (a, b, c etc.) was added to the article number as illustrated in Supplementary Table 1. All publications were independently assessed by groups of two assessors, and separately for each measurand and/or subgroup in the same publication. In the BIVAC, each study is appraised with regards to 14 different quality items (QI), including the assessment of pre-analytical procedures, the measurement procedure, applied statistical methods and the presentation of data [7]. The QI may be assigned either an A, B, C or D score, indicating increasing non-compliance. Based on the individual scores for each of the 14 QI, an overall BIVAC grade is given, equal to the lowest score given to any of the QI. When there was disagreement on the score for any QI between the two assessors, all involved assessors reviewed the study until consensus was reached.

The results of the BIVAC review were registered in an Excel-file (Microsoft), together with data on the study population, study design characteristics and associated BV data (see Supplementary Table 2, exemplified for cystatin C). Confidence intervals (CI) for CV_I and CV_G were calculated from the available data at a 95% probability level, using the method described by Burdick [10] and Sahai [11], when the necessary data on the mean number of subjects and samples and estimates of CV_A were reported.

The global CV_I and CV_G estimates were obtained using a meta-analysis approach, which takes into consideration the inverse of each study’s CI and weights based on the BIVAC grade; i.e. for an A grade paper multiplied by 4, B grade by 2 and C grade by 1 [5]. When publications reported separate estimates for different subgroups, these estimates were first combined to provide a common estimate by applying the weighted mean on the point estimates and corresponding CIs. For the global CV_I, a percentile bootstrap approach [12] was used to indicate measures of uncertainty.

Only studies reporting BV estimates from healthy adults (age range 18–75 years) were included in the meta-analysis. Further exclusion criteria were applied which included: studies in which CI could not be calculated, sampling intervals shorter than twice per week or greater than one month, and studies only including two samples per subject. Additionally, papers were excluded if they had greatly differing estimates derived from non-standard methodologies (i.e. for albumin–electrophoretic separation, for electrolytes–dry chemistry).

Results

In total, 61 BV publications on kidney markers were included. The highest number of published studies were identified for creatinine (n=48) (Table 1). Overall, 259 BV estimates were identified, when included for all analytes and the different subgroups (Table 1). Of the 61 papers reviewed, only three received a BIVAC A grade [13], [14], [15] (Table 1), all of which are recent papers published according to the BIVAC. Four papers received a B grade [16], [17], [18], [19] and the majority, 48 papers, were awarded a C grade. The most common reasons for C scores were, in order of frequency, the following: failure to report homogeneity of variances (homoscedasticity, QI 10), failure to report the number data points used to calculate the BV estimates (QI 12), and failure to report and perform appropriate testing for outliers (QI 8). Five papers (110, 220, 270, 271, 309) were given a D score because the time between phlebotomies was not standardized (QI 3) or multiple analyzers were used throughout the study (QI 4). One study was not appraised as it did not report numerical BV estimates (288). For each analyte, CV_I estimates from all the different subgroups with 95% CI for the point estimate are presented (Figures 1–6). The number of studies available to be included in the meta-analysis ranged from three for cystatin C to 11 for creatinine (Table 2). For several of the measurands, in particular for creatinine, meta-analysis derived estimates appeared slightly lower than those reported in the historical BV database [6] (Table 2). However, direct comparisons are difficult due to the lack of measures of uncertainty for the estimates reported in the historical database. Smaller differences were observed for the CV_G estimates (Table 2).

Figure 1:

CV_I estimates for albumin. Confidence intervals (CI) for individual papers are shown as error bars around the CV_I data point.

Numbers on the x-axis represent the reference id number in the EFLM BV database (as listed in Supplementary Table 1) and letters different subgroups. 95% CI limits for the global BV estimate are shown in green.

Figure 2:

CV_I estimates for creatinine.

Confidence intervals (CI) for individual papers are shown as error bars around the CV_I data point. Numbers on the x-axis represent the reference id number in the EFLM BV database (as listed in Supplementary Table 1) and letters different subgroups. 95% CI limits for the global BV estimate are shown in green.

Figure 3:

CV_I estimates for creatinine ranked according to sample frequency, based on studies from healthy subjects.

In green the upper and lower confidence interval for the meta-analysis derived estimate for the CV_I of creatinine. Values of x-axis are ordered according to sampling interval. D: day, W: week; m: month.

Figure 4:

CV_I estimates for cystatin C. Confidence intervals (CI) for individual papers are shown as error bars around the CV_I data point. Numbers on the x-axis represent the reference id number in the EFLM BV database (as listed in Supplementary Table 1) and letters different subgroups. 95% CI limits for the global BV estimate are shown in green.

Figure 5:

CV_I estimates for electrolytes. Confidence intervals (CI) for individual papers are shown as error bars around the CV_I data point. Numbers on the x-axis represent the reference id number in the EFLM BV database (as listed in Supplementary Table 1) and letters different subgroups. 95% CI limits for the global BV estimate are shown in green.

Figure 6:

CV_I estimates for urea.

Confidence intervals (CI) for individual papers are shown as error bars around the CV_I data point. Numbers on the x-axis represent the reference id number in the EFLM BV database (as listed in Supplementary Table 1) and letters different subgroups. 95% CI limits for the global BV estimate are shown in green.

Discussion

The availability of robust and contextually relevant BV estimates is in many ways an essential requirement for delivery and effective application of clinical laboratory results. The BIVAC may be an important improvement in the assessment of BV studies compared with previous approaches [20]. This is because it specifies essential criteria to be met in reported studies to produce reliable estimates of BV. In addition, BIVAC provides a framework that will help those planning and performing BV studies in the future to produce valid estimates of BV that are transportable into clinical practice across populations. Furthermore, they will be immediately suitable to be included as a fully BIVAC compliant study in the online BV database published by the EFLM [9].

Application of BIVAC to historical publications indicates many fundamental issues that are often not addressed. As an example, the classical method for estimating components of BV put forward by Fraser and Harris [21] required testing the raw data for outliers and variance homogeneity prior to further analysis; many authors studying BV appear not to have addressed this requirement. Users may not be aware of these issues, but they are clearly identified and simply communicated when BIVAC is applied to deliver a grading against 14 QIs that address fundamental requirements, as is done in this systematic review. This review identified only three recent BIVAC grade A papers describing BV of analytes important in the diagnosis, management and monitoring of kidney disease. All the other included studies were graded B, C or D because of failure to comply with one or more of the BIVAC 14 QIs. Most of the included studies in our review had been performed in healthy individuals, whereas a smaller number of studies had addressed BV in different health states and age groups, as detailed for all analytes in the following.

Conclusions

Extensively researched measurands such as the electrolytes, creatinine and urea have produced many papers with comparable values for BV components. The recently published BIVAC compliant EuBIVAS grade A study has confirmed the accuracy of these data, and further A studies do not appear necessary. However, for cystatin C for instance, more data are required.

Some data indicate that BV estimates may be different in some groups of non-healthy subjects as compared to the healthy population. However, proving equality between a group of healthy and non-healthy subjects may require a fully BIVAC compliant study that directly compares both groups.

There are few studies of BV in children and the elderly for most of the measurands. This and differing clinical scenarios are drivers for delivery of appropriately designed BIVAC compliant BV studies for a range of old and new measurands; targets should include elderly, children, specific disease states, analytes in clinical scenarios where within day BV and/or long term BV are required, analytes with few, conflicting or no associated BV data.

Clinicians and Laboratory Medicine Specialists should recognize BV data as reference data that might vary in quality and applicability to their local populations and clinical scenarios. Many BV estimates are the result of well characterized rigorous studies reproduced by multiple groups around the world, but many target measurands appear to require more extensive study. The recent publication of the new EFLM BV database [9] updated with newly published studies, combined with the future delivery of new studies that appropriately powered and BIVAC compliant will provide a useful tool for better healthcare by enabling effective application of appropriately specified laboratory test.