The multicenter European Biological Variation Study (EuBIVAS): a new glance provided by the Principal Component Analysis (PCA), a machine learning unsupervised algorithms, based on the basic metabolic panel linked measurands

Introduction

Estimating biological variation (BV) has many applications: it is used for the setting of the analytical performance specification (APS) for both internal and external quality control [1], [2], [3]; for the assessment of significance of change in serial measurements in a subject (reference change value; RCV); and for determining the number of samples needed to estimate the homeostatic setting point [4, 5]. Moreover, BV estimates are used to assess the utility of conventional population-based reference intervals through the use of the individuality index, and to derive personalized reference intervals, which can improve diagnostic accuracy and treatment appropriateness on a more patient-centered level [6].

The importance of BV estimates, the within-subject BV (CV_I) and the between-subject BV (CV_G), in the last years has been put forward through several activities carried out by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) [7].

As an outcome of the first Strategic Conference of the EFLM held in Milan in November 2014, the EFLM Task Force and Finish Group on BV database (TFG-BVD), was established with the objective to appraise the quality of BV data that are publicly available [8]. Furthermore, responding to the need for reliable BV estimates delivered from big high-quality studies [9, 10], the EFLM WG-BV has designed and implemented the European Biological Variation Study (EuBIVAS) [11, 12].

The design of a multicenter study, aimed at obtaining reliable BV data, requires the involvement of many individuals and a large number of blood collections to provide adequate statistical power, as demonstrated by Røraas et al. [13]. The EuBIVAS is a high-power study, which included 91 apparently healthy volunteers from 5 European countries, from which high-quality BV data, based on the newest generation of analytical methods, have been derived for a set of measurands [12, 14].

However, the multicenter project thus conceived posed the risk of introducing factors impacting on the reliability and hence on the applicability of the obtained BV data. Namely, the two most important aspects were: considering the enrolled population as homogeneous, even if coming from different countries; and introducing a significant pre-analytical variability, having performed the collections in several centres.

To minimize these latter aspects, a very detailed protocol was prepared, agreed upon, and rigorously followed by each involved laboratory in all steps [14], by taking into account all the requirements of the EFLM checklist for BV studies [15, 16].

Previously published EuBIVAS papers [17], [18], [19], [20] reported no significant differences in pre-analytical variables or treatment between the different laboratories. However, these previous works were focused on the analysis of specific set of measurands, without a comprehensive general look that could take into consideration multi-variate information coming from a panel of several measurands, possibly together with some characteristics of the population (i.e. lifestyle).

Machine learning (ML) is a methodological approach developed within the field of artificial intelligence (AI) aimed at developing algorithms and models by which to make classifications and predictions on the basis of large amounts of data taken as examples to “learn” relevant patterns and schemas. The essence and main novelty of ML methods is to infer new sets of rules or new algorithms from data to perform complex cognitive tasks (like diagnosis or prognosis), without step-by-step instructions explicitly given in advance by the programmer [21]. ML is an umbrella term to denote a large variety of statistical methods, which include supervised and unsupervised approaches. In the former case, labelled data (data containing both the input and the output variables) are given to the learning algorithm to automate classification (predicting a categorical variable) or regression (predicting a continuous variable) tasks. By contrast, in the unsupervised case, only input data are given, and the goal is to try to modelling the underlying structure or distribution of the data, and to find unknown patterns. Among unsupervised algorithms, Principal Component Analysis (PCA) is one of the most widely used methods. In short, PCA is a techniques for dimensionality reduction that is an approach by which multidimensional data can be reduced into fewer, new dimensions that embed some information from the original ones. In particular, PCA describes the standardized multidimensional data in terms of components that are linear combinations of the original variables preserving the variance of the original data. For these reasons, PCA can also be applied to easily and meaningfully visualize high dimensional data in case the first 2 o 3 principal components are chosen, that is those that explain a maximal amount of variance.

In any case, whether supervised or unsupervised, accurate predictions by ML algorithms rely on data completeness and quality [21], according to the principle commonly described as “garbage in, garbage out”.

In particular, the presence of outliers, or extreme values, in a dataset may result in poor predictive modelling performance. However, identifying and removing outliers can be a very challenging and time consuming task, even with a good understanding of the structure of the data. Several strategies to detect outliers are available, including univariate and multivariate methods. With univariate methods, each variable is considered independently, and a value is labelled as an outlier when it is too far (according to some criteria) from a central tendency indicator. However, these methods usually fail in detecting observations that deviate from global behavior or from the pattern of the majority of data (with pattern here we indicate the relationship between multiple variables of the dataset) [22].

In this work, by applying PCA, we evaluated the homogeneity of EuBIVAS data through a novel approach, with the goal of detecting previously unknown patterns or relationships between variables. Moreover, traditional univariate approach to detect outliers was compared to the multivariate approach. With this aim, the EuBIVAS measurands included in the basic metabolic panel (BMP), glucose (Glu), calcium (Ca) [17], albumin (Alb), total protein (TP) [18], sodium (Na), potassium (K), chloride (Cl), urea, total bilirubin (TBil) [17], creatinine (Crea) [19], phosphatase alkaline (ALP), aspartate aminotransferase (AST), alanine aminotransferase (ALT) [20], and some characteristics of the population (alcohol intake, smoking habits, physical activity, body mass index (BMI) [11, 23], [24], [25] have been considered.

Materials and methods

Databases

To investigate a ML approach to detect the outliers from the original EuBIVAS raw dataset, three different databases have been considered, as described below and summarized in Figure 1.

Figure 1:

a flow chart describes the three databases (DB1, DB2 and DB3).

The flow chart describes the process to obtain three databases: the initial number of results, the number of outliers detected, the number of missing values replaced and the final number of data considered for database 1 (DB1), database 2 (DB2) and database 3 (DB3) respectively.

The EuBIVAS dataset included 22,280 raw data values, instead of the theoretical 23,660 (13 measurands, 10 samples in duplicate, 91 subjects): 5.8% of data were missing, since only 77 participants completed all 10 collections; while 10 participants completed nine collections, two participants completed eight collections, and further two participants completed seven collections [11, 12].

Database 1

The original database, used to publish the BV estimates EuBIVAS based, was used as the reference database or database 1 (DB1). DB1 consisted of 21, 712 results, obtained after the application of statistical tests for outlier detection (568 data values), as previously described in EuBIVAS publication [17], [18], [19], [20].

As a first step, the missing data were imputed by means of the missing procedure (see below “Missing data”), to obtain the total theoretical amount of 23,660 data (91 subjects, 13 measurands, 10 collections, two replicates). As a second step, the values beyond 4 standard deviations, for each measurand distribution (at least one replicate compared to the general distribution of all subjects), were discarded. Finally, the DB1 was imputed a second time using the missing procedure.

Database 2

The original database of 22,280 EuBIVAS raw data values was used as a starting point to obtain the database 2 (DB2). As a first step, the missing data were imputed, to obtain the total theoretical amount of 23,660 data. As a second step, the values beyond 4SD, for each measurand distribution, were discarded. As a third step, the missing data was imputed a second time using the missing procedure (Figure 1).

Subsequently, a multivariate outlier detection technique, called Elliptic Envelope [26], was applied. Elliptic Envelope is based on the robust covariance estimation of the data. This method assumes that data is sampled from a multivariate Gaussian distribution (which can be visualized in the multi-dimensional space as an ellipse), so that outlier observations are defined as those observations lying “too far” outside of the ellipse fitted around the data. The outlier-ness of the observations is determined through the Mahalanobis distance. Briefly, the Mahalanobis distance is the distance between a multidimensional point and a distribution, computed as d(x) = [(x−µ)^T · C⁻¹ · (x−µ)]^0.5, where x is the vector of the observations, µ the vector of mean values, T denotes matrix transposition, and C⁻¹ the inverse covariance matrix [22, 26]. Compared with traditional univariate outlier detection approaches, this method, being based on a multivariate criterion, takes into account, at the same time, all results obtained from the same sample (two replicates for 13 measurands). Therefore, for each outlier detected 26 results were discarded (one sample). As a final step, the DB2 was imputed a second time, using the missing procedure, to obtain the total theoretical amount of 23,660 data (Figure 1).

Database 3

The last database, database 3 (DB3), is the most conservative database. It differs from the DB2 in that the last missing procedure is not performed. By contrast, missing results are discarded from further analysis. Thus, the final amount of data included in DB3 is lower than that included in DB1 and DB2, and it corresponds to the data obtained after having applied the Elliptic Envelope ML technique (Figure 1).

Results

DB1. The number of outliers detected and removed by the traditional statistical analysis for the DB1 was previously published [17], [18], [19], [20], while the number of the imputed missing values, together with the number of the second outlier detection procedure, are reported in Figure 1.

DB2 and DB3. The flowchart (Figure 1) reports the number of the imputed missing values, the number of observations detected as outliers beyond the 4SD, and the number of observations (samples) detected as outliers by the Elliptic Envelope technique (multivariate procedure), for both DB1 and DB2. Fifty-one observations were detected as outliers as beyond the 4SD from the general distribution, while the assessment of the Elliptic Envelope ML technique led to the exclusion of further 91 samples (10% of the total set of data). These 91 samples have been removed from 49 subjects, resulting in a mean of 1.9 observations/subject.

A subject without any data for a given measurands never occurred, so that the imputation of 20 values randomly generated with the same mean values and DS of the other subjects was not necessary.

For each database, the total number of imputed missing values, for each measurand, is reported in Table 2.

The PCA visualizations obtained for the three databases (panel a, b and c), for the mean and CV_P values, are reported in Figures 2 and 3 respectively. The subjects are represented through different colors, according to the gender/age groups. A clear clustering has been identified between males and females mean values (Figure 2) with an accuracy of 86, 86 and 85% for DB1, DB2, and DB3 respectively, while for CV_P values no clustering has been found (accuracy around 60% in all databases) (Figure 3). Interestingly, the points that identify the mean values of the females in menopausal age, are plotted closer to the male subgroup. Data interpretations derived from the three different databases are similar. The feature importance scores, computed by means of the PCA analysis, for the three different databases for the mean values clustering are shown in supplemental figure 1. ALP, followed by Crea, ALT and AST, resulted as the most significant features for all databases.

Figure 2:

Mean values gender related 2 principal component analysis (2PCA).

Each point in the picture represents a single subject position in a space in a two dimensional representation (2PCA) obtained using 13 mean values as features. The blue, pink and red circles indicate the men, females in fertile and in menopausal age respectively. The data from database 1 (DB1), database 2 (DB2), and database 3 (DB3), are represented in the three different panels (A, B and C, respectively).

Figure 3:

CV_P values gender related 2 principal component analysis (2PCA).

Each point in the picture represents a single subject position in a space in a two dimensional representation (2PCA) obtained using 13 CV_P values as features. The blue, pink and red circles indicate the men, females in fertile and in menopausal age respectively. The data from database 1 (DB1), database 2 (DB2), and database 3 (DB3), are represented in the three different panels (A, B and C, respectively).

As for the gender/age subgroups, no significant differences have been found, in the three databases, also for country of origin, smoking habits, alcohol intake, BMI and physical activity (data not shown). For these latter, PCA visualization are shown only for the DB3.

No clustering related to the country of origin was detectable (Figure 4), for both mean values (Figure 4A), and CV_PS values (Figure 4B). Figure 5 reports the PCAs for the physical activity (panels a, b) and for the BMI (panels c, d). A negligible clustering (accuracy 63%) appears for the mean values according to the BMI (panel c). Similarly, irrelevant clusterings for smoking habits, and alcohol intake are shown in Figure 6, for both mean and CV_P values.

Figure 4:

Mean and CV_P values countries related 2 principal component analysis (2PCA).

Each point in the picture represents a single subject position in a space in a two dimensional representation (2PCA). The different colors represent the country of origin: light blue for Italy, black for Norway, orange for Spain, yellow for Turkey, and green for The Netherland. The mean values and CV_P data are represented in panels (A) and (B), respectively.

Figure 5:

Mean and CV_P values according to physical activity and BMI 2 principal component analysis (2PCA).

Each point in the picture represents a single subject position in a space in a two dimensional representation (2PCA). Panels (A) and (B): different colors represent different amounts of physical activity: none in blue, medium in red (<7 h/week), and strenuous in black (>7 h/week) for mean values and CV_P data (panel a and b respectively). Panels (C) and (D): body mass index (BMI) < 21 and BMI > 21 for mean and CVP values are represented in black and in light blue in (C) and (D), respectively.

Figure 6:

Mean and CV_P values according to alcohol consumption and smoking habits 2 principal component analysis (2PCA).

Each point in the picture represents a single subject position in a space in a two dimensional representation (2PCA). Panels (A) and (B): different colors represent different smoking habits: none in green, low in blue (<5 cigarettes/day), and high in black (>5 cigarettes/day) for mean values and CV_P data (panel a and b respectively). Panels (C) and (D): different colors represent different alcohol intake: none in light blue, moderate in pink (<2 Units/day), and high in red (>2 Units/day) for mean values and CV_P data (panels (A) and (B), respectively).

Discussion

Although ML is routinely used in biochemical development and for evaluating and interpreting data in several medical specialties, the application of ML in clinical laboratory medicine still appears to be an unexpectedly slow process [28]. However, “A short guide for medical professionals in the era of artificial intelligence” has recently been published, with the aim to serve as a short repository of information and details every physician might need to know in the age of AI [29]. Laboratory data are influenced by several factors including the pre-analytical (collection, time, stability, storage temperature, etc.) [30], and the analytical phase (standardization, harmonization etc.) [31, 32], therefore they should be used with caution and competence, in collaboration with clinicians [21].

Hereby, we applied ML techniques on a metabolic panel of measurands from the EUBIVAS study, with a dual purpose: first, to verify the homogeneity of the involved population, coming from different European countries; second, to evaluate the existence of possible clusterings, due to different lifestyles, and to validate a new multivariate approach to detect outliers using ML techniques. For this purpose, the EuBIVAS database, used in previously publication [17], [18], [19], [20], cleaned of the outliers detected by the traditional univariate statistical tests, was used, and compared to other two databases obtained as described in the previous sections. Furthermore, since most ML algorithms cannot be applied on datasets with missing data, a missing data imputation procedure was applied. In this respect, it is worth noting that the missing procedure here conceived, has been established to do not alter the mean and variance values either in the single subject, or in the subgroup, or in the whole population.

The number of data removed by the multivariate procedure to obtain the databases 2 and 3 (Figure 1) is of 2,366 results (10% of the total number of data), much higher than the number of data (568 data, 2.4%) removed by the traditional statistical tests used individually for each measurand. This difference in the number of outliers identified and removed should not surprise, if we considered that multivariate methods can identify observations which deviate from the global behavior of the data (that is, observations that have relationships between variables which are different from most of the other observations), even in the absence of extreme values in each single variable (univariate approach).

Despite the different methods applied to detect the outliers, and the differences among the number of data removed, the visualizations obtained applying PCA analysis to the three databases were similar: this confirms the face validity of the ML multivariate method used to remove the outliers, and of the missing replacement procedure.

Figure 2 shows an evident clustering in all databases, based on the mean values for males and females subgroups. This clustering, here identified, means that with an accuracy of 85% it is possible to predict, from laboratory data, whether the subject is male or female. Such a relevant clusterization, using mean values of the measurands here considered, should not be considered surprising as it is obviously due to the different values concentration between males and females for some measurands, which is also reflected in different reference intervals (ALP, creatinine, ALT and AST). Interestingly, these measurands were identified as the most important features for the PCA clustering (supplemental figure 1). However, as shown in Figure 2, it is interesting to note that the same clustering has been found in all databases with almost the same accuracy (between 85 and 86%), confirming the fact that the databases are able to provide the same information, despite different approaches in missing values identification and replacement. This further suggests that observations identified as outliers by the multivariate method were also labelled as outliers by the univariate method. Moreover, this expected clustering with gender indirectly confirms that the PCA approach works. Another noteworthy finding, regards the subgroup of menopausal females that are overall displaced from the female area into male area, indicating the need to set appropriate reference intervals for females in menopausal age, for which more studies are desirable. The six males that are moved in females area (mean age 34.8 ± 3.3 years), three from Italy and three from Turkey, are characterized to be sedentary (5 out 6 declared to do not do any physical activity, and just one only light activity), to be drinkers (6 out of 5 more than 2U/day of alcohol consumption) and no smokers (just one out of 6 declared to smoke), and to have a high BMI (mean 26.3 ± 2.6).

Contrary to that observed for the mean values, the CV_P values obtained in males’ and females’ subgroups, do not show any visible clustering in any database (Figure 3). All this seems to suggest that, despite the differences in mean values, the component of BV is not overall affected by the concentration gender-related, confirming the validity in using a unique RCV and APS values based on BV estimates.

Considering that the three databases have always given overlapping results (data not shown), for convenience, PCAs related to the data of the only DB3 are shown in the next figures.

The countries of origin of the subjects enrolled in EuBIVAS are highlighted by different colors in Figure 4 for mean (a) and CV_P values (b) respectively. In Figure 4, no clustering for the country is recognizable, which implies that it is not possible to distinguish any sub-population from their origin and, as a consequence, the whole EUBIVAS population can be considerable homogenous. Actually, EuBIVAS studied previously published about the measurands here included, found a significant difference in population only in the case of creatinine mean values for the Turkish people [19], while for other measurands no differences among countries were found.

The observation that the mean creatinine concentration was lower in Turkish participants is not surprising. This has been reported previously, being explained as resulting from differences in diet (low meat consumption) and different physical activity [19]. This was also confirmed by the lifestyle information provided by the enrolled subjects in EuBIVAS (Table 1). In this study, the small difference in creatinine concentration, is “hidden” in a multi-measurand panel, so that the EuBIVAS population all together can be considerate suitable, being homogenous, for having estimated BV generizable data. This last consideration further stresses the importance that both the multivariate as well as the univariate approach may give valuable information.

From the PCAs visualization (Figure 4), there is no data suggesting any differences in pre-analytical variables or samples treatment among the different laboratories. This finding is particularly relevant, considering the criticality in the pre-analytical phase of some measurands included in the metabolic panel. For example, several pre-analytical quality indicators in laboratory medicine are strongly dependent on the time the sample is collected, and careful attention to the pre-analytical phase is essential to ensure accurate glucose and electrolytes measurements for the delay in specimen processing [33], [34], [35]. Moreover, many factors can affect analyte stability for most laboratory tests. Just think about the enzyme activities measurements, where sampling time and storage information are crucial for qualifying a sample as suitable for being tested [33, 36].

Conclusions

The EuBIVAS protocol followed a strict experimental design [11, 12], powered to deliver estimates of BV with a high degree of reliability for a well-characterized multinational cohort of subjects.

The absence of meaningful differences between groups from Turkey, Norway, The Netherlands, Spain, and Italy confirms that the obtained data are widely applicable across healthcare systems and that they can be used to deliver APS for systems to be used internationally.

Similarly to what has been observed for the country variable, the absence of clusterings for alcohol, smoking habits, BMI and physical activity further confirms that the EuBIVAS sample is homogenous.

In conclusion, our data support the use of ML PCA technique and of the multivariate approach to detect outliers, in alternative to univariate methods, to gain new insight of the data. Moreover, this novel approach confirms the homogeneity of the EuBIVAS dataset.