Towards the rational utilization of SARS-CoV-2 serological tests in clinical practice

To the Editor,

Coronavirus disease 2019 (COVID-19), caused by the severe acute respiratory syndrome coronavirus (SARS-CoV-2), continues to be a pandemic. The development of nucleic acid-based tests, in particular real-time reverse transcription polymerase chain reaction (rRT-PCR), has allowed a more accurate diagnosis of the disease to be made, particularly in asymptomatic patients [1]. The development of serological assays, which measure the antibody responses induced by SARS-CoV-2, may improve disease management, but the limited knowledge gained on antibody kinetics, seroconversion rates and correlation with clinical severity are major factors to be evaluated for appropriate adoption of serological testing in clinical practice [2]. Several authors described the analytical validation of available SARS-CoV-2 assays, as well as their diagnostic accuracy [3], [4]. However, from a clinical viewpoint, the data reported on sensitivity and specificity fail to provide actionable information for either diagnosis or patient management. To determine whether an individual is immune to, or infected by, SARS-CoV-2, we need to know pre-test probability in the specific population being tested, since disease prevalence strongly affects the predictive positive value (PPV). This is also recommended by the US Centers for Disease Control and Prevention (CDC), in an interim guideline [5]. To evaluate the effect of disease prevalence on serological tests, we used the following commercially available chemiluminescent immunoassays (CLIA) for IgG SARS-CoV-2 antibodies: (a) Maglumi (New Industries Biomedical Engineering Co. Ltd [Snibe], Shenzhen, China), (b) Liaison (Diasorin S.p.A. Saluggia [VC], Italy), and (c) iFlash (Shenzhen Yhlo Biotech Co. Ltd. China) in 151 subjects (64 SARS-CoV-2 patients and 87 healthcare workers) who underwent at least one nasopharyngeal swab test, analyzed by RT-PCR. For SARS-CoV-2 patients the mean time interval from the onset of symptoms and serological assessment was 24 days (SD ± 11; range 12–54 days). Of the 87 healthcare workers, 71 were considered negative, since at least three sequential molecular test results were negative, and the remaining 16 were considered positive, with mild disease. For each assay we selected two different thresholds to maximize overall performance (Youden index) and specificity (fixed at 95%), respectively.

Table 1 shows the PPV and negative predictive value (NPV) at three different disease prevalence settings: (a) 4%, as found in a Veneto Region (Italy) survey (data not shown); (b) 10%, as described in a survey conducted in Geneva [6] and (c) 23%, as reported in a pediatric dialysis unit [7]. While NPV is elevated in all three cases (>94.8%), PPV ranges from 21.1% in the area with the lowest (4%), to 85.7% in the area with highest (23%) prevalence. Figure 1 shows the effect of pre-test probability in reducing uncertainty in diagnosing COVID-19 disease, using the Fagan nomogram and the data from one of the assays evaluated (Liaison). For the other two assays, the effect was similar, thus indicating that only at high disease prevalence does PPV provide valuable and actionable information.

Figure 1:

Fagan nomogram to show the pre-test probability in reducing uncertainty in diagnosing COVID-19 disease using the Liaison assay with two different diseases prevalence.

On the left (A) 4% prevalence; on the right side (B) 23% prevalence.

Our data confirm that serological assays are of utmost importance, not only for understanding the prevalence of and immunity against SARS-CoV-2, but also for ruling out suspected disease, being NPV always elevated. Diagnosis according to serology can only be effective in clinical settings with high estimated prevalence. Ideally, a diagnostic test should have excellent performance in both detecting and ruling out the disease, with both a high sensitivity and specificity. However, in the real world this is often a “mission impossible”, calling for optimization in both detection and exclusion of disease. Thus, it is often necessary to define the purpose of the test (detection or exclusion), and calculate the best possible threshold for maximizing sensitivity or specificity. High-quality serological assays have recently been developed and adopted. Yet, this is no time for complacency: the ‘take home’ message is that we must urgently face the new, pressing challenge of applying and deploying these tests in a rational manner. Any failure to do so will seriously undermine diagnostic precision, and quality of care, consequently endangering individuals, and communities at large.