Where is laboratory medicine headed in the next decade? Partnership model for efficient integration and adoption of artificial intelligence into medical laboratories

Janne Cadamuro

The laboratory specialists focus on the analytical phase of the total testing process, aiming for higher quality, shorter turn around time (TATs), increased analytical accuracy, has led clinicians to think about the laboratory as a results producing facility. When categorizing all process steps within the pre-analytical, analytical and post-analytical phases into ‘medical’, ‘organizational’, and ‘analytical’, and combining these categories with the focus of most laboratories, it becomes evident that we have lost sight of the medical part of our profession (Figure 1). As mentioned above, contributing to patient care individually in unfeasible, which is why AI systems are needed aiding in test selection and interpretation. When fed some ground-truth data (data for the diagnostic work-up of the symptom or disease of interest, including all medical information, laboratory test results, and final diagnosis), these systems are able to calculate the most efficient and effective diagnostic pathway [14]. However, taking all variables into account, such as the patients pre-conditions, medications, physical examination, anamnesis, other diagnostic data and of course laboratory test results, is far more complex and error prone than image recognition models used in other medical disciplines. Using automated systems in medical care with limited accuracy levels may have devastating consequences. Jovičić et al. evaluated Mobile health applications, aiding patients in the interpretation of laboratory data and compared these with the opinions of laboratory professionals and confirmed the low utility of currently available laboratory medicine apps [15].

Figure 1:

Categories of laboratory processes and the corresponding labs emphasis.

Therefore, in all the euphoria, we have to be cautious and careful and weigh the risks and benefits of such systems. Reports, such as the Cambridge Analytica scandal or the failed million-dollar project IBM Watson, are very good examples for how such systems may be misused or overrated [16, 17].

AI algorithms, aiding laboratory professionals in test selection and result interpretation will most probably not only make our profession more efficient, but more medical. Clinicians would be able to focus on their core expertise and laboratory professionals would answer clinical questions instead of reporting numbers. Therefore, the laboratory specialist should be open for new technology, rather than fearing the unknown.

Clare Weir

(The opinions in this paragraph are expressly the view of the author and do not constitute the opinions or ideas of their employer).

Patient pathway and tailored diagnostics are becoming more prevalent in the clinical field [18]. This ensures that the correct diagnosis and therapy is provided to the patient rather than a one for all system, which may not fit to the patient’s history, symptoms and diagnosis. At this moment in time AI and ML is in its infancy in clinical laboratory practice and will probably not become mainstream for some years to come [19].

Working through the patient pathway from the first visit to a General Practitioner to monitoring of therapy in a haemato-oncology setting, there is only one area at the moment where ML and AI is applied and that is within the morphology section during the diagnostic process in both the haematology laboratory and flow cytometry laboratory (Figure 2). However, this does not mean that the morphologist is taken out of the picture; they are still required to validate the overall results.

Figure 2:

Illustrating the patient pathway from the perspective of a leukaemia or lymphoma^a. ^aPicture provided by Dr. Marion Münster, Sysmex Europe SEU.

In haematology, usually rule based algorithms are programmed into the middleware rather than AI models. These rules provide information on suspected abnormalities and provide suggestions of interpretive comments, possible reflex tests or alarms in case of critical values to the laboratory specialist. Although applying AI models to this area would be useful, the complexity and number of variables is currently too high to reach satisfying predictive values.

In flow cytometry, the use of samples such as bone marrow, cerebro-spinal fluid, synovial fluids and solid tumour tissue has to be acknowledged. Diagnostic errors by an AI/ML model may have devastating consequences, like insufficient, wrong or unnecessary treatment.

In an ideal setting, patient cases are discussed collaboratively among clinicians and laboratory specialists, each providing valuable expertise. Patient-specific suggestions, provided by AI/ML models, may improve this process in the future; however, currently this task is too complex to be done validly by such systems.

Finally, the patients’ point of view needs to be considered from a psychological perspective. Medicine heavily relies on person-to-person interactions. Patients want to be treated as individuals, not as numbers or statistical data, which is why all professions in the medical field are one of the least likely to be atomized in the future [20].

Raj Gopalan

As mentioned above, although modern laboratories have become incredibly accurate and fast in sample processing and result delivery, test selection and result interpretation by clinicians continue to use a manual cognitive process, making them prone to human error and variation (Figure 3). The quality of this process depends on the clinician’s training, experience, and diagnostic skills to determine the right differential diagnosis for the patient based on history, signs, symptoms, and physical examination. After a list of provisional differential diagnoses is formulated, clinicians must tap into their knowledge of diagnostics in laboratory medicine, as well as imaging, to choose the appropriate set of tests to help rule in and rule out the diagnoses on the list as well as to identify potential risks for future manifestation of specific diseases, such as cancer, liver or kidney diseases, or sepsis. When interpreting the results, clinicians must consider the contributions of the combined meaning of each of the laboratory test parameters and their clinical significance. Just scanning for abnormal result flags is not sufficient, since it does not assess important biochemical processes that may indicate underlying diseases. These tasks can be challenging, as the medical school educational curriculum does not typically include coursework on modern technology, such as AI and machine learning, to help future doctors manage these complex multidimensional problems [21, 22].

Figure 3:

The brain-to-brain loop concept for laboratory testing^a. ^aAdapted from Lundberg GD. Acting on significant laboratory results. JAMA. 1981;245:1762–3.

This results in many diagnostic errors and billions of dollars of malpractice claims each year [23, 24]. Therefore, the question remains: “Do clinical laboratories have a role in controlling skyrocketing healthcare costs in a post-COVID era? Can they help improve individual and population health by enabling early and accurate disease identification, clinical diagnosis, and treatment?” [24]. Even though laboratories’ efficiency, accuracy, and consistency have improved over the years, delivering decision support to physicians, predominantly, has not expanded much beyond providing reference intervals and highlighting results that fall outside of those intervals [25]. Some laboratories are additionally providing delta-checks and reflex-testing algorithms.

Although these capabilities offer definitive value to clinicians, they are basic compared to those that laboratories can potentially offer. In recent times, and especially with the advent of molecular and genetic testing, the number of choices for laboratory testing has skyrocketed. The Choosing Wisely initiative started by the American Board of Internal Medicine (ABIM) to help clinicians order the appropriate tests is endorsed by more than 70 medical specialities. The goal of this initiative is to improve the safety and quality of patient care and reduce harm [26]. With the advent of electronic medical records (EMR), institutional laboratories have access to the initial, provisional diagnoses documented by clinicians. It seems logical that a future service provided by the laboratory would include recommendation of an optimal test battery that could be used to help to rule in the most probable diagnosis, rule out the unlikely ones, and identify the risks for life-threatening diseases. In addition, it may be possible to rank-order tests that are less expensive and covered by the patient’s insurance. AI and ML technology can be used not only in the test-selection phase but also to help interpret multidimensional test results, providing likelihood scores that can be used to identify the risks for potential diseases and provide probability scores for subclinical diseases the patient may have, based on patients with a similar demographic and lab-test result profile from a large population of confirmed diagnosed cases [27]. CDSS can play a crucial role in tracking blood results and trends to monitor organ functions, thereby ensuring appropriate dosing to maintain patients within the most effective therapeutic spectrum. Thus, CDS creates the opportunity to pull data from multiple modalities, IVD, imaging, and the EMR; integrate that data; and present it visually to the multispecialty care team to enable comprehensive assessment of patients for optimal management decisions.

As we enter the post-COVID era, it is imperative to expand the traditional role of the clinical laboratory beyond its four walls to further benefit hospital units and clinics [28]. In addition to the decision-support information and services the laboratory can provide, there is also opportunity for more-direct engagement by laboratory specialists.

Laboratory directors who are trained physicians specializing in laboratory medicine have a unique role in working with other clinical specialists on care teams to evaluate, diagnose, and treat patients. With the advent of the EMR system and data interchange standards, laboratory directors can access complete medical data for patients and have the expertise to help clinicians order appropriate tests, interpret results, and guide further investigations.

Federico Cabitza

Laboratory medicine is no exception to the usage of computational classification systems created utilizing ML techniques in medicine for a variety of use cases and applications, such as diagnosis, prognosis, and risk stratification. However, though common and widely used in other domains, classification performance measures based on the concept of error (such as accuracy or the C-statistic) are unlikely to be meaningfully applicable to medicine.

Three primary reasons for distinct approaches for the evaluation of the efficacy of these decision support systems are here advocated. The first reason is the issue of replicability. Accuracy estimations are based on historical data with features that are often obtained from a single (or a few) institutions involved in the development of the ML model. Several studies have found that when used in different circumstances, even very accurate models report relevant drops in their accuracy [29]. The models must then be validated externally, using data from a diverse set of sources that are distinct (in terms of work habits and equipment) from those participating in the creation process [30, 31]. To achieve this goal, sound data similarity metrics must become more common and widely applied, allowing researchers to determine whether validation data are similar or different from training and test data, and thus whether accuracy scores are strongly correlated with similarity or not: this means focussing on robustness rather than accuracy. The second factor is the issue of noise [32]. Laboratory data are often thought of as good, reliable, and structured data; and rightly so, especially when compared to other types of health data, such as clinical or patient-reported data, which are influenced by observer variability to an extent that is often underestimated, if not completely ignored. ML processes, on the other hand, nearly entirely ignore the phenomena of analytical and biological variability [33]. This calls for data augmentation techniques and methods for creating synthetic data that are more indicative of the overall phenotypic complexity (and variability) that defines the circumstances for which the system is asked to provide trustworthy advice or prediction. This is also a matter of robustness.

The third issue is that of meaning. Traditional error-based metrics distinguish between false-negative and false-positive errors, but they do not differentiate between instances on any dimension, such as relevance, diagnostic difficulty, rarity, or whether accurate results were also found to be interpretable by experts. Furthermore, these metrics do not weigh results in terms of prediction risk (that is, confidence score): as a result, estimates are produced by assuming not only that the cases are all equivalent for the sake of simplicity, but also that the decisions are all equal, even though they are based on very different probability estimates (and often with unknown or low calibration).

All of this calls for the creation of more reliable and comprehensive utility measurements that include predictive calibration. It is necessary to focus on interpretability, robustness, and utility rather than accuracy as a performance indicator in order to make AI support more reliable and trustworthy.

Jochen K. Lennerz

As discussed in the previous sections, AI has a huge potential to disrupt laboratory medicine soon. However, euphoria and promises cannot replace demonstration of clinical utility. It is noteworthy that clinical utility definitions vary across settings and use-cases [34]. When considering diagnostic testing – the promises of AI can be summarized as improvements in diagnostic testing quality. However, how can we capture the difficult topic of diagnostic test quality? Dr Lennerz proposed a nested conceptualization of diagnostic test quality that can be applied to the assessment and quantification of the quality impact when implementing AI models (Figure 4).

Figure 4:

Conceptualization of a diagnostic AI model in the healthcare ecosystem. Each diagnostic test (first layer) is part of a specific set of operations collectively referred to as diagnostic procedure (second layer). Each laboratory typically has numerous diagnostic procedures. The diagnostic procedures interface with the external health care delivery system (third layer, diagnostic service layer). The diagnostic quality can be seen as the combination of the diagnostic quality of the diagnostic test, procedure, and service. Considering the deployment of AI models, the diagnostic quality impact of an AI model can be expressed as the absolute (abs.) difference between the quality with or without the AI model. For simplicity of the conceptualization, the AI model is exemplarily depicted in the diagnostic test layer; however, AI models can be implemented in other and/or multiple layers.

The conceptual starting point is the consideration that the AI model is integrated into the existing healthcare ecosystems. In the simplest case, the AI model (a computer program) becomes part of a specific diagnostic test (or a component of a test). The integration of the model as part of the diagnostic test is considered the first layer (=diagnostic test layer). Once accomplished, the model becomes an integral part of this specific laboratory testing process (also known as the laboratory procedures). The procedure that now entails an AI model is the second layer. With very few exceptions, most laboratories have multiple diagnostic testing procedures, and these may entail multiple models with their individual use cases, value propositions, and performance metrics (=value add). Each of these testing procedures is facing outward – externally towards the physician and the patient, which can be considered a diagnostic service. The diagnostic service offered by the laboratory (to a hospital or to outside partners) is considered the third layer.

In this conceptualization, the diagnostic quality is a composite of diagnostic test, diagnostic procedure, and diagnostic service. Consequently, the quality impact of AI models should be considered a function of the improvements related to the diagnostic test(s), the diagnostic procedure(s) and the diagnostic service(s) with their various intended use cases and value propositions.

How does this relate to regulatory issues? Now there are major challenges when encountering the portfolio of these hundreds and numerous elements that AI solution may touch. Among the major challenges and risks, issues regarding privacy, security, fairness, transparency and explainability, safety and performance, bias or third-party risks need to be considered [35, 36]. Regulations are complex and touch upon numerous aspects that culminate in compliance [37]. Documents and regulations in the field of AI regulation are manifold in and evolving [38, 39]. It is important to point out that regulations are made – and the field that challenges current regulation through the application of the scientific method (i.e., data) is called regulatory science [40]. Specifically, Regulatory science is the scientific discipline that evaluates and challenges current regulation; benefit vs. risk assessment, and submission/approval strategies. Regulatory science uses a distinct terminology, aiming to describe what (intended use), who and why is doing something (indication of use), where and when it is done (context of use) and how is done (instructions of use). These definitions ultimately define the stringency during validation (=acceptance testing) and ongoing performance assessments (=proficiency testing) in clinical practice.

We have moved from diagnostics to advanced diagnostic technologies – we have now reached data science and AI. Where do we go next? We need to learn how to apply our high-quality data efficiently using computational technologies. However, when the complexity exceeds human capabilities, we need tools that help us streamline these divers functions and govern their risk and safety profiles. These tools include meaningful regulations. In other words, what is next in laboratory medicine? We should consider the ability to actively influence laboratory regulation as an emerging field in laboratory medicine. Rather than receiving the rules, we can actively contribute, share our knowledge, and help shape our field.