Skip to content
BY 4.0 license Open Access Published by De Gruyter April 7, 2021

Lumipulse G SARS-CoV-2 Ag assay evaluation using clinical samples from different testing groups

  • Giulia Menchinelli , Licia Bordi , Flora Marzia Liotti , Ivana Palucci , Maria Rosaria Capobianchi , Giuseppe Sberna , Eleonora Lalle , Lucio Romano , Giulia De Angelis , Simona Marchetti , Maurizio Sanguinetti EMAIL logo , Paola Cattani and Brunella Posteraro

Abstract

Objectives

Compared to RT-PCR, lower performance of antigen detection assays, including the Lumipulse G SARS-CoV-2 Ag assay, may depend on specific testing scenarios.

Methods

We tested 594 nasopharyngeal swab samples from individuals with COVID-19 (RT-PCR cycle threshold [Ct] values ≤ 40) or non-COVID-19 (Ct values >40) diagnoses. RT-PCR positive samples were assigned to diagnostic, screening, or monitoring groups of testing.

Results

With a limit of detection of 1.2 × 104 SARS-CoV-2 RNA copies/mL, Lumipulse showed positive percent agreement (PPA) of 79.9% (155/194) and negative percent agreement of 99.3% (397/400), whereas PPAs were 100% for samples with Ct values of <18 or 18–<25 and 92.5% for samples with Ct values of 25–<30. By three groups, Lumipulse showed PPA of 87.0% (60/69), 81.1% (43/53), or 72.2% (52/72), respectively, whereas PPA was 100% for samples with Ct values of <18 or 18–<25, and was 94.4, 80.0, or 100% for samples with Ct values of 25–<30, respectively. Additional testing of RT-PCR positive samples for SARS-CoV-2 subgenomic RNA showed that, by three groups, PPA was 63.8% (44/69), 62.3% (33/53), or 33.3% (24/72), respectively. PPAs dropped to 55.6, 20.0, or 41.7% for samples with Ct values of 25–<30, respectively. All 101 samples with a subgenomic RNA positive result had a Lumipulse assay’s antigen positive result, whereas only 54 (58.1%) of remaining 93 samples had a Lumipulse assay’s antigen positive result.

Conclusions

Lumipulse assay was highly sensitive in samples with low RT-PCR Ct values, implying repeated testing to reduce consequences of false-negative results.

Introduction

Antigen testing has recently been added to the landscape of clinical laboratory methods to detect and combat the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which is the notorious cause of coronavirus disease 2019 (COVID-19) (https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antigen-tests-guidelines.html#anchor_1597523027400). Like the molecular – relying on real-time reverse-transcription polymerase chain reaction (RT-PCR) and, to date, the standard method for the etiological COVID-19 diagnosis–antigen testing detects the presence of SARS-CoV-2 in the acute infection phase only [1].

Theoretically, antigen-based assays are advantageous in terms of fast turnaround times and reduced costs but are less sensitive than RT-PCR-based assays [2]. Additionally, the former have the disadvantage to provide false-positive results, which leads false-positive patients to be managed as patients with true SARS-CoV-2 infection [3, 4], unless that a positive antigen result is rapidly confirmed by molecular testing. To mitigate this issue, the European Centre for Disease Prevention and Control (ECDC) recommends antigen-based assays to be not only carefully selected but also validated before their implementation in clinical practice [5]. Since June 2020, the Lumipulse G SARS-CoV-2 Ag (Fujirebio, Tokyo, Japan), detecting SARS-CoV-2 nucleocapsid (N) protein, is being used in Japan where a positive antigen test result is enough to definitively diagnose COVID-19 without PCR [4] – which is instead mandatory in European countries to confirm positive antigen results [5]. Two independent studies by Hirotsu et al. [6, 7] reported on the performance of the Lumipulse G SARS-CoV-2 Ag assay (hereafter referred as the Lumipulse assay) using nasopharyngeal swab samples. In both studies, samples with high viral load (corresponding to low values of RT-PCR cycle threshold [Ct] – an accredited measure of virus [5]) or samples collected in the early infection phase showed complete concordance between Lumipulse and RT-PCR results. At the time of current study’s submission, Hirotsu et al. [8] released the results of a study performed on nasopharyngeal swab samples to validate the Lumipulse assay clinically, whereas Basso et al. [9] evaluated the Lumipulse assay for use with self-collected saliva samples.

With the aim to fully understanding its usefulness, we evaluated the Lumipulse assay with 594 individuals’ nasopharyngeal swab samples assigned to different testing groups (i.e. including early or late infection patients). To this end, we compared Lumipulse assay antigen results with those of RT-PCR assay targeting SARS-CoV-2 genomic RNA (usually used as an indicator of viral presence [2]). In parallel, RT-PCR positive samples were analyzed for the presence of subgenomic RNA (recently proposed as an indicator of active viral replication [10]) to support Lumipulse assay’ results.

Materials and methods

Study design and clinical samples

This study was conducted at the Fondazione Policlinico Universitario A. Gemelli IRCCS (FPG). We included nasopharyngeal swab samples from patients/individuals (≥18-year aged) presenting at and/or admitted to our institution during a two-week period in December 2020. Samples were from laboratory-confirmed COVID-19 (n=194) or non-COVID-19 (n=400) diagnoses, which relied, respectively, on positive (Ct values of ≤40) or negative (Ct values of >40) results obtained using the Seegene Allplex 2019-nCoV, the DiaSorin Simplexa COVID-19 Direct, or the Roche Diagnostics Cobas SARS-CoV-2 Test RT-PCR assays [11], [12], [13]. For example, the Seegene Allplex 2019-nCoV assay is a single-tube assay targeting the envelope (E), RdRP (RNA-dependent RNA polymerase), and N SARS-CoV-2 genes and running on a Bio-Rad CFX96 Real-time Detection system. Based on Ct values – i.e. numbers of cycles the fluorescent signal crosses the threshold for positive detections – the Seegene software automatically analyzes RT-PCR results. By this assay, a Ct value ≤ 40 for at least one of two viral genes (i.e. RdRP and N) or for the E gene alone indicates, respectively, the certain or presumptive presence of SARS-CoV-2 RNA in the sample. No positive samples only for E gene were included in the study. In view of relatively lower performance of the DiaSorin Simplexa COVID-19 Direct assay [11, 14], samples (n=39) initially tested with this assay were retested with the Seegene Allplex 2019-nCoV assay to confirm (positive) results. Likewise, samples with discordant results between the RT-PCR and the Lumipulse assays (see below) were confirmed as positive (n=39) or negative (n=3) by retesting as previously described [15].

For stratification purposes [5], we selected positive samples based on their Ct values (i.e. 11.2–39.9) to include samples with different viral load levels. These samples were characterized for the inclusion in three testing groups, namely diagnostic, screening, and monitoring groups, which were in substantial accordance with the definitions reported in the interim technical guidance by the Centers for Disease Control and Prevention (CDC) for rapid SARS-CoV-2 antigen testing (https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antigen-tests-guidelines.html#anchor_1597523027400). Accordingly, diagnostic or monitoring groups included persons who had signs or symptoms (i.e. clinical illness) consistent with COVID-19, who had no clinical illness but a recent known or suspected exposure to SARS-CoV-2, or who had a previous laboratory-confirmed COVID-19 diagnosis, whereas the screening group included persons who were asymptomatic and without known or suspected exposure to SARS-CoV-2. In particular, persons in diagnostic or monitoring groups were patients admitted to the emergency department and/or COVID-19 wards of the FPG hospital, whereas persons in the screening group were hospital’s or other workplace’s employees or were students, faculty, or staff of the hospital affiliated Università Cattolica del Sacro Cuore (Rome, Italy). All positive samples were further stratified in five groups based on RT-PCR Ct values (<18, 18–<25, 25–<30, 30–<35, and 35–40).

All samples originally collected in 3 mL of universal transport medium (UTM; Copan, Brescia, Italy) were portioned in aliquots that were kept at 4 °C until testing with the Lumipulse assay (see below), which was always performed within 2–4 h from the time samples were subjected to RT-PCR for detecting SARS-CoV-2 genomic RNA as above described. In parallel, additional aliquots from the same samples were frozen at −80 °C until testing for SARS-CoV-2 subgenomic RNA (see below). Furthermore, we used archived frozen samples (RT-PCR negative) as a matrix to generate contrived samples for the Lumipulse assay’s analytical sensitivity determination (see below).

Lumipulse assay for SARS-CoV-2 antigen detection

The Lumipulse assay quantitatively detects SARS-CoV-2 N protein in clinical samples (e.g. nasopharyngeal swab) by a specific two-reaction chemiluminescence-based immunoassay method on the Lumipulse G1200 automated immunoassay analyzer (Fujirebio). In the first reaction, the sample (or the SARS-CoV-2 Ag calibrator) and the sample treatment solution are added to an anti-SARS-CoV-2 monoclonal antibody-coated magnetic particle solution, and then incubated for 10 min at 37 °C to allow formation of specific antigen-antibody immunocomplexes. In the second reaction (accessed after washing), an alkaline phosphatase-labelled anti-SARS-CoV-2 monoclonal antibody solution is added and incubated for 10 min at 37 °C to allow specific binding to the antigen of aforementioned immunocomplexes, and then to form additional immunocomplexes. Finally (after washing), a substrate solution is added and incubated for 5 min at 37 °C, and the resulting chemiluminescence signals are automatically read by the analyzer and used to calculate the SARS-CoV-2 antigen’s amount in the sample through the interpolation with a SARS-CoV-2 Ag calibrator curve.

We determined the limit of detection (LOD) of the Lumipulse assay according to a previously described protocol [12]. Briefly, aforementioned contrived samples were spiked with a dilution series of Vero E6 cell-cultured SARS-CoV-2 (INMI-1 strain) at a concentration range of 1.0 × 105 50% tissue culture infective dose (TCID50)/mL (4.0 × 108 RNA copies/mL) to 1.0 TCID50/mL (4.0 × 103 RNA copies/mL), and then tested in replicates (Supplementary Figure S1). For each sample, SARS-CoV-2 RNA was amplified by RT-PCR in Rotor-GeneQ Real-Time cycler (Qiagen, Hilden, Germany), using the RealStar SARS-CoV-2 RT-PCR kit 1.0 (Altona Diagnostic GmbH, Hamburg, Germany). RNA copies/mL were calculated through a standard curve prepared with serially diluted EURM-019 single-strand SARS-CoV-2 RNA fragments (https://crm.jrc.ec.europa.eu/p/EURM-019). Thus, we plotted the probability (y-axis) against the SARS-CoV-2 concentration’s logarithm (x-axis), and we calculated the 95% LOD value, which was the lowest concentration at which the replicates yielded positive detection 95% of the time (Supplementary Figure S1).

Before testing with the Lumipulse assay, samples were centrifuged at 3,000×g for 15 min to allow separation of the supernatant from the remaining viscous UTM material, and 100 µl were analyzed for the antigen quantification as above described. Results were interpreted using a cutoff of 1.34 pg/mL as established by the Lumipulse assay’s manufacturer, and were expressed as negative (<1.34 pg/mL), gray-zone positive (1.34–10 pg/mL), or positive (>10–>5,000 pg/mL) results, respectively. At first, we diluted samples with an antigen level exceeding the detection limit (i.e. 5,000 pg/mL), and dilutions were used to quantify the original samples’ antigen levels based on the dilution factor. Next, this was no longer performed because of ensuing logistic issues (e.g. reagent shortage). Thus, antigen concentrations >5,000 pg/mL were rounded to 5,000 pg/mL for all the samples.

RT-PCR assay for SARS-CoV-2 subgenomic RNA detection

To determine the presence of SARS-CoV-2 subgenomic RNA (i.e. E gene subgenomic RNA), samples were subjected to a previously developed in-house RT-PCR assay [16]. This is an adaptation from the method described by Wölfel et al. [10] that looks specifically at the E gene subgenomic RNA to indicate active virus infection/transcription [17]. Briefly, SARS-CoV-2 RNA (also including genomic RNA) was extracted from samples using the Seegene Nimbus automated system and then used for the RT-PCR assay. This was performed with the Qiagen OneStep RT-PCR kit (Qiagen, Valencia, CA) and a 25-μL reaction volume containing 600 nM concentration each of primers (sgE_SARS-CoV2_F 5′-CGATCTCTTGTAGATCTGTTCTC-3′; sgE_SARS-CoV2_R 5′-ATATTGCAGCAGTACGCACACA-3′) and 200 nM concentration of probe (sgE_SARS-CoV2_P 5′-FAM-ACACTAGCCATCCTTACTGCGCTTCG-BBQ-3′). Thermal cycling consisted of 30 min at 50 °C for reverse transcription, followed by 15 min at 95 °C and subsequent 45 cycles each of 10 s at 95 °C, 15 s at 55 °C, and 30 s at 72 °C.

Data collection and analysis

Data were presented as numbers with percentages or as means ± standard deviation (SD), as appropriate. To determine Lumipulse assay’s LOD, the MedCalc statistical software (MedCalc Software Ltd, Ostend, Belgium) was used to convert RT-PCR positive detection proportion into a “probability unit” (or “probit”). Lumipulse assay’s results were categorized as positive, gray-zone positive, or negative and, then, compared using a one-way analysis of variance (ANOVA) with the Tukey’s multiple-comparison test. For Lumipulse assay’s or subgenomic RNA assay’s results, differences between a priori established groups were assessed using the chi-square test or the Student’s t-test, as appropriate. Percent agreement values, with their respective confidence intervals (CIs), were calculated comparing Lumipulse assay’s or subgenomic RNA assay’s results with those obtained by the reference method (i.e. genomic RNA RT-PCR assay). Correlation between antigen levels (as determined by the Lumipulse assay) and Ct values (as determined by the reference method) was assessed using the Spearman’s correlation coefficient. Statistical analysis was conducted using Stata 15 (StataCorp, College Station, TX, USA) or GraphPad Prism 7 (GraphPad Software, San Diego, CA, USA) software, and p<0.05 was considered statistically significant.

Results

SARS-CoV-2 antigen (Lumipulse assay) vs. genomic or subgenomic RNA (RT-PCR assay) testing

First, we determined the analytical capability of the Lumipulse assay, a recently marketed assay for SARS-CoV-2 N protein detection in European countries. As shown in Supplementary Figure S1, the LOD was 2.95 TCID50/mL, corresponding to 1.2 × 104 SARS-CoV-2 RNA copies/mL, at 95% detection probability. Then, 594 nasopharyngeal swab samples, including RT-PCR positive (n=194) or negative (n=400) samples, were tested with the Lumipulse assay.

(i) Overall performance: Using SARS-CoV-2 RNA genomic RT-PCR assay as the reference method (Table 1), the Lumipulse assay detected 155 of 194 samples as positive (antigen concentration, ≥1.34 pg/mL) and 397 of 400 samples as negative (antigen concentration, <1.34 pg/mL). This resulted in a positive percent (PPA) of 79.9% (95% confidence interval CI 73.6–85.3) and a negative percent agreement (NPA) of 99.3% (95% CI 97.8–99.8), respectively. Of 155 samples, 29 (18.7%) were positive within the gray-zone (antigen concentration, 1.34–10 pg/mL), which defines an antigen positivity extent necessitating to be confirmed by RT-PCR. As depicted in Figure 1 and detailed in Table 1, we stratified Lumipulse assay’s results according to RT-PCR Ct values. Thus, we found significant differences in the mean Ct value ± SD for 126 samples with antigen-positive results (21.95 ± 6.03) as compared to 29 samples with antigen (gray-zone)-positive results (30.85 ± 3.19) and to 39 samples with antigen-negative results (33.79 ± 2.39), respectively (ANOVA with Tukey’s multiple-comparison test; p<0.0001 for both comparisons) (Figure 1). Interestingly, PPAs between Lumipulse assay’s and RT-PCR assay’s results were 100% for samples with Ct values of <18 (n=38) or 18–<25 (n=49) and 92.5% for samples with Ct values of 25–<30 (n=37). For 31 of 155 samples with Ct values of 30–<35 (n=23) or 35–40 (n=8), PPAs dropped to 47.9 and 42.1%, respectively. More interestingly, 24 (82.8%) of 29 antigen (gray zone)-positive results regarded samples with Ct values ranging from 25 to 35, whereas three (100%) of three antigen (gray zone)-positive results regarded (antigen false-positive) samples with Ct values > 40.

Table 1:

Comparison of Lumipulse assay results with the RT-PCR assay results stratified by Ct values for SARS-CoV-2 detection.

RT-PCR assayaLumipulse assayb
Ct values (no. of results)No. of positive (including gray-zone positive) resultsPercent agreement (95% confidence interval)
≤40 (194)155 (29)79.9 (73.6–85.3)
<18 (38)38 (0)100.0 (90.7–100.0)
18–<25 (49)49 (1)100.0 (92.7–100.0)
25–<30 (40)37 (10)92.5 (79.6–98.4)
30–<35 (48)23 (14)47.9 (33.3–62.8)
35–40 (19)8 (4)42.1 (20.3–66.5)
>40 (400)3 (3)99.3 (97.8–99.8)
  1. RT-PCR, real-time reverse-transcription polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; Ct, cycle threshold. aSARS-CoV-2 RT-PCR testing was performed on 594 individuals’ nasopharyngeal swab samples. A positive result (i.e. a Ct ≤ 40) for at least one of two viral targets with the Seegene Allplex 2019-nCoV (N [nucleocapsid] and RdRP [RNA-dependent RNA polymerase]), the DiaSorin Simplexa COVID-19 Direct (S [spike] and Orf1ab [open reading frame 1ab]), or the Roche Diagnostics Cobas SARS-CoV-2 Test (E [envelope] and ORF1a) assays indicated the presence of SARS-CoV-2 RNA in individual’s nasopharyngeal swab samples (n=194). Samples with a Ct > 40 for the mentioned genes were considered negative (n=400). bSARS-CoV-2 antigen testing was performed on 594 individuals’ nasopharyngeal swab samples with the Lumipulse G SARS-CoV-2 Ag assay. Using the manufacturer’s cutoff of 1.34 pg/mL, results were interpreted as gray-zone positive or positive when antigen concentrations were 1.34–10 and >10–5,000 pg/mL, respectively.

Figure 1: Distribution of Lumipulse assay’s results according to RT-PCR assay’s Ct values in 194 nasopharyngeal samples.Results are presented as positive (>10–5,000 pg/mL), gray-zone positive (1.34–10 pg/mL), or negative (<1.34 pg/mL), respectively. In each violin plot, solid line indicates the mean Ct value (21.95, 30.85, and 33.79, respectively) and the area between dotted lines indicates the standard deviation value (6.03, 3.19, and 2.39, respectively). Asterisks indicate statistically significance (p<0.0001) between result groups, as established using a one-way analysis of variance (ANOVA) with the Tukey’s multiple-comparison test.
Figure 1:

Distribution of Lumipulse assay’s results according to RT-PCR assay’s Ct values in 194 nasopharyngeal samples.

Results are presented as positive (>10–5,000 pg/mL), gray-zone positive (1.34–10 pg/mL), or negative (<1.34 pg/mL), respectively. In each violin plot, solid line indicates the mean Ct value (21.95, 30.85, and 33.79, respectively) and the area between dotted lines indicates the standard deviation value (6.03, 3.19, and 2.39, respectively). Asterisks indicate statistically significance (p<0.0001) between result groups, as established using a one-way analysis of variance (ANOVA) with the Tukey’s multiple-comparison test.

(ii) Performance by different testing groups: Table 2 shows the results of 194 antigen-positive samples – overall described in Table 1 – stratified by the diagnostic (n=69), screening (n=53), or monitoring (n=72) groups of testing for 194 patients with laboratory-confirmed COVID-19 diagnosis. Only for the monitoring group, samples used in the study were not the same as those at the COVID-19 diagnosis time; thus, this group included COVID-19 patients who were tested during the course of disease. Conversely, 122 patients in the two remaining groups were tested at early disease phases. Table 2 also shows the results from SARS-CoV-2 subgenomic RNA detection that was performed in parallel on the 194 samples.

Table 2:

Positive detections of Lumipulse antigen and subgenomic RNA compared with those of RT-PCR for SARS-CoV-2 in different testing groups.a

Ct values (no. of results)Results according to indicated RT-PCR Ct values for:p-Valued
Lumipulse antigen detectionbSubgenomic RNA detectionc
No. of results (including gray-zone results)Percent agreement (95% confidence interval)No. of resultsPercent agreement (95% confidence interval)
Diagnostic group
<18 (15)15 (0)100.0 (78.2–100.0)1386.7 (59.5–98.3)0.14
18–<25 (20)20 (0)100.0 (83.2–100.0)20100.0 (83.2–100.0)NA
25–<30 (18)17 (1)94.4 (72.7–99.9)1055.6 (30.8–78.5)0.009
30–<35 (10)6 (5)60.0 (26.2–87.8)00.0 (0.0–30.8)0.003
35–40 (6)2 (0)33.3 (4.3–77.7)116.7 (0.4–64.1)0.52
All (69)60 (6)87.0 (76.7–93.9)4463.8 (51.3–75.0)0.002
Screening group
<18 (15)15100.0 (78.2–100.0)15100.0 (78.2–100.0)NA
18–<25 (17)17100.0 (80.5–100.0)1694.1 (71.3–99.9)0.30
25–<30 (10)8 (5)80.0 (44.4–97.5)220.0 (2.5–55.6)0.007
30–<35 (8)3 (2)38.0 (8.5–75.5)00.0 (0.0–36.9)0.05
35–40 (3)00.0 (0.0–70.6)00.0 (0.0–70.6)NA
All (53)43 (7)81.1 (68.0–90.6)3362.3 (47.9–75.2)0.03
Monitoring group
<18 (8)8100.0 (63.1–100.0)8100.0 (63.1–100.0)NA
18–<25 (12)12 (1)100.0 (73.5–100.0)1083.3 (51.6–97.9)0.13
25–<30 (12)12 (4)100.0 (73.5–100.0)541.7 (15.2–72.3)0.002
30–<35 (30)14 (7)46.7 (28.3–65.7)13.3 (0.1–17.2)<0.001
35–40 (10)6 (4)60.0 (26.2–87.8)00.0 (0.0–30.8)0.003
All (72)52 (16)72.2 (60.4–82.1)2433.3 (22.7–45.4)<0.001
  1. RT-PCR, real-time reverse-transcription polymerase chain reaction; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; Ct, cycle threshold; NA, not applicable. aGroups were established according to the Centers for Disease Control and Prevention (CDC) definitions for testing settings (https://www.cdc.gov/coronavirus/2019-ncov/lab/resources/antigen-tests-guidelines.html#anchor_1597523027400), and were further stratified by viral load (i.e. Ct values) as indicated. bSARS-CoV-2 antigen was detected in nasopharyngeal swab samples of groups’ individuals by the Lumipulse G SARS-CoV-2 Ag assay, which provides a 0.01–5,000 pg/mL measurement range. Using the manufacturer’s cutoff of 1.34 pg/mL, results were expressed as negative, gray-zone positive, or positive when antigen concentrations in the samples were <1.34, 1.34–10, or >10–5,000 pg/mL, respectively. Samples with antigen concentrations above 5,000 pg/mL were rounded to 5,000 pg/mL for convenience reasons. cSARS-CoV-2 subgenomic RNA was detected in nasopharyngeal swab samples of groups’ individuals by an in-house RT-PCR assay for the presence of replicative (E gene) RNA. dFor comparisons between percent agreement rates.

Regarding antigen detection results, PPA with the reference method (i.e. SARS-CoV-2 RNA genomic RT-PCR assay) was 87.0% (95% CI 76.7–93.9; 60/69 results), 81.1% (95% CI 68.0–90.6; 43/53 results), or 72.2% (95% CI 60.4–82.1; 52/72 results) in diagnostic, screening, and monitoring groups, respectively. Consistent with that shown in Table 1, PPA was 100% for samples with Ct values of <18 or 18–<25 in all three testing groups, and was 94.4, 80.0, or 100% for samples with Ct values of 25–<30 in diagnostic, screening, and monitoring groups, respectively. Regarding subgenomic RNA detection results, PPA with the reference method was 63.8% (95% CI 51.3–75.0; 44/69 results), 62.3% (95% CI 47.9–75.2; 33/53 results), or 33.3% (95% CI 22.7–45.4; 24/72 results) in diagnostic, screening, and monitoring groups, respectively. Unlike antigen detection results, PPAs for samples with Ct values of <18 or 18–<25 were, respectively, 86.7 and 100% in the diagnostic group, 100 and 94.1% in the screening group, and 100 and 83.3% in the monitoring group. Interestingly, in all three groups, PPAs dropped to 55.6, 20.0, or 41.7% for samples with Ct values of 25–<30, and reached 0% for almost all samples with Ct values of 30–<35 or 35–40, respectively. A chi-square test analysis was conducted to compare PPAs between antigen and subgenomic-RNA detections among the three testing groups, and this analysis revealed significant differences for the samples overall (p=0.002, p=0.03, and p <0.001, respectively) or the samples with Ct values ranging from 25–<30 or 35–40 (p<0.05 for all comparisons).

(iii) Correlation between antigen levels and RT-PCR Ct values: To corroborate these findings, we assessed antigen levels in relation with the SARS-CoV-2 viral load expressed as RT-PCR Ct values. A Spearman’s correlation analysis was conducted for all 194 samples that tested positive with the RT-PCR assay, which were analyzed according to aforementioned testing groups (i.e. diagnostic, screening, and monitoring). As shown in Figure 2, we found a significant (negative) association between antigen levels and Ct values in either diagnostic (Spearman’s ρ=−0.82; p<0.0001), monitoring (Spearman’s ρ=−0.76; p<0.0001), or screening (Spearman’s ρ=−0.72; p<0.0001) groups. As it can see, association was relatively stronger in the diagnostic group and less strong in the screening group.

Figure 2: Correlation between the SARS-CoV-2 antigen levels quantified by the Lumipulse assay and the SARS-CoV-2 genomic RNA Ct values obtained with the RT-PCR assay. Analysis was separately conducted for (A) diagnostic, (B) screening, and (C) monitoring groups of testing. Antigen concentration is expressed as log10 pg/mL. Concentrations of <1.34, 1.34–10, and >10 pg/mL were used to interpret Lumipulse assay’s antigen results as negative, gray-zone positive, or positive, respectively.
Figure 2:

Correlation between the SARS-CoV-2 antigen levels quantified by the Lumipulse assay and the SARS-CoV-2 genomic RNA Ct values obtained with the RT-PCR assay. Analysis was separately conducted for (A) diagnostic, (B) screening, and (C) monitoring groups of testing. Antigen concentration is expressed as log10 pg/mL. Concentrations of <1.34, 1.34–10, and >10 pg/mL were used to interpret Lumipulse assay’s antigen results as negative, gray-zone positive, or positive, respectively.

Relationship between SARS-CoV-2 antigen and subgenomic RNA

To investigate this issue, we analyzed the characteristics of 194 antigen-positive or -negative samples according to the presence (n=101) or absence (n=93) of subgenomic RNA. As shown in Table 3, all 101 samples with a subgenomic RNA positive result had a Lumipulse assay’s antigen positive result, whereas only 54 (58.1%) of remaining 93 samples had a Lumipulse assay’s antigen positive result. Samples in the subgenomic RNA-positive group had a mean Ct value ± SD – at the RT-PCR assay for genomic RNA – that significantly differed from that of samples in the subgenomic RNA-negative group (20.3 ± 4.8 vs. 29.9 ± 4.8; Student’s t-test; p<0.001). Likewise, the time from COVID-19 diagnosis to testing (mean days ± SD) for samples in the subgenomic RNA-positive group significantly differed from the time for samples in the subgenomic RNA-negative group (1.6 ± 3.3 vs. 6.1 ± 7.0; Student’s t-test; p<0.001). In total, 93 (47.9%) of 194 genomic RT-PCR assay positive samples tested negative with the subgenomic RT-PCR assay, and 39 (41.9%) of 93 samples also tested negative with the antigen Lumipulse assay. As detailed in Figure 3, distribution of the 39 samples varied across testing groups, being 51.3% (20 samples), 25.6% (10 samples), and 23.1% (9 samples) in monitoring, screening, or diagnostic groups, respectively.

Table 3:

Characteristics of 194 RT-PCR positive samples that tested positive or negative for the presence of SARS-CoV-2 subgenomic RNA.a

CharacteristicbSamples with a subgenomic RNA positive result (n=101) grouped asSamples with a subgenomic RNA negative result (n=93) grouped asp-Valuec
RT-PCR positive/Antigen positive (n=101)RT-PCR positive/Antigen negative (n=0)RT-PCR positive/Antigen positive (n=54)RT-PCR positive/Antigen negative (n=39)
RT-PCR Ct, mean value ± SD20.3 ± 4.8NAd29.9 ± 4.833.8 ± 2.4<0.001
Testing from COVID-19 diagnosis, mean days ± SD1.6 ± 3.3NAd6.1 ± 7.07.7 ± 8.6<0.001
  1. aAll samples were from diagnostic (n=69), screening (n=53), or monitoring (n=72) testing groups (see Table 2). Testing for SARS-CoV-2 subgenomic RNA was performed using an in-house RT-PCR assay to assess the presence of replicative (E gene) RNA. bThe time period between SARS-CoV-2 RT-PCR (to which Ct values refer) used to diagnose COVID-19 and testing for SARS-CoV-2 subgenomic RNA (and antigen) ranged from 0 days in the diagnostic or screening groups to 32 days in the monitoring group. Only in the last group, consequently, two temporally different samples were tested. cFor comparisons between the RT-PCR positive/Antigen positive groups herein listed. dNA, not applicable.

Figure 3: Distribution of Ct results by SARS-CoV-2 genomic or subgenomic RT-PCR assays for 194 samples across three groups (diagnostic, screening, and monitoring) of testing.In each group, samples that tested positive or negative with the Lumipulse antigen assay are shown.
Figure 3:

Distribution of Ct results by SARS-CoV-2 genomic or subgenomic RT-PCR assays for 194 samples across three groups (diagnostic, screening, and monitoring) of testing.

In each group, samples that tested positive or negative with the Lumipulse antigen assay are shown.

Discussion

Using RT-PCR as the best available comparator method [1], we showed that the Lumipulse assay had PPA (sensitivity) and NPA (specificity) of ∼80 and 99%, respectively. The number of false-negative and false-positive results driven by these metrics was 39 (among 194 RT-PCR positive results) and 3 (among 400 RT-PCR negative results), respectively (Table 1). As the Lumipulse assay returns positive results as positive (>10–5,000 pg/mL) or gray-zone positive (≥1.34–10 pg/mL), it is worth noting that all the three false-positive results fell within the gray-zone range. These results would have been confirmed by RT-PCR if antigen testing had been performed as a frontline diagnostic method at the study time.

In our hands, Lumipulse assay met the minimum performance requirements of ≥80% sensitivity and ≥97% specificity for rapid antigen tests as established by the World Health Organization (https://www.who.int/publications/m/item/covid-19-target-product-profiles-for-priority-diagnostics-to-support-response-to-the-covid-19-pandemic-v.0.1) and, later, agreed by the ECDC [5]. Additionally, we showed that the Lumipulse assay’s sensitivity increased from ∼93 to 100% with samples that displayed RT-PCR Ct values below 25–30. As a reflection of high viral load, these values are likely associated with an infectious SARS-CoV-2, contrasting higher Ct values (>30–40) that, instead, are likely associated with a non-infectious SARS-CoV-2 [10]. We tempted to explain the apparently low performance of the Lumipulse assay as compared to RT-PCR (i.e. ≥90% sensitivity [https://www.who.int/publications/m/item/covid-19-target-product-profiles-for-priority-diagnostics-to-support-response-to-the-covid-19-pandemic-v.0.1]) by assessing the Lumipulse assay’s results with respect to the results of SARS-CoV-2 subgenomic RNA assay [10]. Among 194 (SARS-CoV-2 genomic RNA) RT-PCR positive samples, 101 (52.1%) samples had positive results for subgenomic RNA (and antigen), with 82 (94.3%) of 87 antigen-positive samples having RT-PCR Ct values below 18–25 (Tables 2 and 3). Importantly, 54 subgenomic RNA negative but Lumipulse antigen positive samples as well as the 101 samples above mentioned were tested after their respective patients had received a COVID-19 diagnosis, which was 6.1 ± 7.0 days in one case and 1.6 ± 3.3 days in the second case (Table 3). These findings confirm that the detection of SARS-CoV-2 subgenomic RNA may occur in diagnostic samples (i.e. respiratory tract swab samples) several days after active virus replication/infection has occurred [17]. Consistently, detection was documented up to 5 days in the Wölfel et al.’s study [10] and up to 11 days in the Alexandersen et al.’s study [17], although the first study (like us) focused on the E gene subgenomic RNA and the second study on the Orf7a subgenomic RNA as detected by PCR. Our study also suggests that SARS-CoV-2 antigen may be longer detected than SARS-CoV-2 subgenomic RNA. Whether this is somewhat indicative of a greater stability of the virion (and its protein component) than the RNA of SARS-CoV-2 needs elucidation. We recall that the SARS-CoV-2 virion does not contain subgenomic RNA – i.e. RNA replicative forms thought to encode the structural spike (S), E, membrane (M), or N virus proteins – whereas subgenomic RNA is part of cellular membrane vesicles and thereby relatively stable [17]. Thus, antigen and subgenomic RNA represent two SARS-CoV-2 biological entities worthy of investigation in clinical samples [7, 18, 19], especially in situations of prolonged (genomic RNA) RT-PCR positivity implying infectious virus shedding [17, 20].

We tempted to fully appraise the Lumipulse assay’s performance by analyzing 194 RT-PCR positive samples stratified by groups of testing (Table 2). Therefore, we included 194 adults suspected of (n=69) or screened for (n=53) SARS-CoV-2 infection or monitored for confirmed COVID-19 (n=72) in diagnostic, screening, or monitoring groups, respectively. We found that the Lumipulse assay worked well, and almost equally, in all three testing groups, with 60 (diagnostic group), 43 (screening group), or 52 (monitoring group) samples being positive. Expectedly, the subgenomic RNA assay yielded positive results in 44 (diagnostic group), 33 (screening group), or 24 (monitoring group) samples. Of note, lowest sample positivity rates were seen in the monitoring group with both Lumipulse (52/72 samples, 72.2%) and subgenomic RNA (24/72 samples, 33.3%) assays. These findings concur with the idea that SARS-CoV-2 antigen or, particularly, subgenomic RNA results are likely to be less positive in monitoring scenarios where positive results for genomic RNA are, instead, indicative of prolonged SARS-CoV-2 shedding [17]. Accordingly, in our monitoring group, the time from COVID-19 diagnosis to testing was longer (up to 32 days) than in diagnostic or screening groups (0 days).

To the best of our knowledge, this is one of largest clinical studies currently evaluating the Lumipulse assay. Compared to previous studies [6, 7], our set of tested nasopharyngeal swab samples is not only wider but also uncharted – we included 594 individuals’ samples from testing scenarios with different pretest probability that, in turn, reflected different clinical situations. Nonetheless, our findings agreed with those by Hirotsu et al. [6] showing that the SARS-CoV-2 antigen levels declined in consecutively collected samples of seven patients from the time of their hospital admission to discharge. Therefore, the finding that antigen positivity rates varied according to whether samples were in a diagnostic/screening rather than in a monitoring scenario reinforces the hypothesis raised by Hirotsu et al. [6] that antigen testing could be also useful to identify patients in the early or late phase of SARS-CoV-2 infection. In the last Hirotsu et al.’s study [8] – that contemporary to our study – Lumipulse assay’s sensitivity was 92.5% (37/40 samples) and specificity was 100% (989/989 samples) compared to the RT-PCR assay. The mean Ct value ± SD in the 37 positive samples was 21.0 ± 4.4. Consistently, in our study, sensitivity of the Lumipulse assay was 100% (49/49) for samples with Ct values of 18–<25 and 92.5% (37/40) for samples with Ct values of 25–<30 (Table 1). In an evaluations’ review of five antigen tests (four commercial and one in-house) by Dinnes et al. [21], average sensitivity was 56.2% (95% CI 29.5–79.8%) and average specificity was 99.5% (95% CI 98.1–99.9%) based on five studies with 943 samples (596 were confirmed SARS-CoV-2 samples). To enhance the applicability of our Lumipulse assay’s results, we determined the assay’s analytical sensitivity (Supplementary Figure S1) before sample testing to ensure that the assay’s LOD was equivalent to 104 viral genomic copies/mL, which is the desirable limit acknowledged until now (https://www.who.int/publications/m/item/covid-19-target-product-profiles-for-priority-diagnostics-to-support-response-to-the-covid-19-pandemic-v.0.1). Furthermore, we limited oversampling of samples with laboratory-confirmed SARS-CoV-2 infection, which accounted for the high risk of bias affecting patient selection in many published studies [21]. Stratifying our study participants by days from the symptom onset [5] was impracticable for us. However, we compensated for this limitation by including testing groups that were comparable for size (∼60 RT-PCR positive samples per group), and we assumed that RT-PCR negative samples were almost equally distributed across testing groups. Nevertheless, further stratification of the samples according to their Ct values resulted in very small subgroups in some instances (Table 2), which might have limited the strength of our findings. Additionally, Hirotsu et al. [6, 8] and we used clinical samples collected in a 3-mL UTM volume, which implies that smaller or larger collection volumes might not perpetuate the current data with the Lumipulse assay. We also acknowledge that differing conditions in the pre-analytical centrifugation step for samples between our study (i.e. 3,000×g for 15 min) and Hirotsu et al.’s studies (1,300×g for 10 min or 2,000×g for 5 min) might have affected the Lumipulse assay antigen detection at a different extent.

To summarize, our results show that Lumipulse assay’s performance was satisfactory, confirming the current view about antigen-based laboratory testing for SARS-CoV-2 detection. In particular, the Lumipulse assay was highly sensitive to detect SARS-CoV-2 antigen in samples with low RT-PCR Ct values (<25) by overall or different testing scenarios. While Ct values > 25 might not correspond to situations with active SARS-CoV-2 infection and/or infectivity, a strategy of repeated testing can maximize the Lumipulse assay’s performance and thereby reduce consequences of false-negative results.


Corresponding author: Maurizio Sanguinetti, MD, Dipartimento di Scienze Biotecnologiche di Base, Cliniche Intensivologiche e Perioperatorie, Università Cattolica del Sacro Cuore, Rome, Italy; and Dipartimento di Scienze di Laboratorio e Infettivologiche, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy, E-mail:
Giulia Menchinelli, Licia Bordi, Flora Marzia Liotti, Paola Cattani, and Brunella Posteraro contributed equally to this work. The order of authors was determined randomly.

Funding source: Reale Group

Funding source: Fondazione Valentino Garavani & Giancarlo Giammetti

  1. Research funding: The authors are grateful to the Reale Group and the Fondazione Valentino Garavani & Giancarlo Giammetti for providing financial support to the COVID-19 Research at the FPG (Fondazione Policlinico Universitario A. Gemelli IRCCS) of Rome (Italy), or with the Italian Health Ministry for providing funds to the INMI (Istituto Nazionale per le Malattie Infettive Lazzaro Spallanzani IRCCS) of Rome (Italy).

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Informed consent was obtained from all individuals included in this study.

  5. Ethical approval: The FPG Ethics Committee approved the study (Reference Number 49978/20).

References

1. Brooks, ZC, Das, S. COVID-19 testing. Am J Clin Pathol 2020;154:575–84. https://doi.org/10.1093/ajcp/aqaa141.Search in Google Scholar

2. Cheng, MP, Papenburg, J, Desjardins, M, Kanjilal, S, Quach, C, Libman, M, et al.. Diagnostic testing for severe acute respiratory syndrome–related coronavirus-2: a narrative review. Ann Intern Med 2020:M20–1301.10.7326/M20-1301Search in Google Scholar PubMed PubMed Central

3. Mak, GC, Lau, SS, Wong, KK, Chow, NL, Lau, CS, Lam, ET, et al.. Analytical sensitivity and clinical sensitivity of the three rapid antigen detection kits for detection of SARS-CoV-2 virus. J Clin Virol 2020;133:104684. https://doi.org/10.1016/j.jcv.2020.104684.Search in Google Scholar

4. Ogawa, T, Fukumori, T, Nishihara, Y, Sekine, T, Okuda, N, Nishimura, T, et al.. Another false-positive problem for a SARS-CoV-2 antigen test in Japan. J Clin Virol 2020;131:104612. https://doi.org/10.1016/j.jcv.2020.104612.Search in Google Scholar

5. European Centre for Disease Prevention and Control (ECDC). Options for the use of rapid antigen tests for COVID-19 in the EU/EEA and the UK. 19 November 2020. Stockholm: ECDC; 2020.Search in Google Scholar

6. Hirotsu, Y, Maejima, M, Shibusawa, M, Nagakubo, Y, Hosaka, K, Amemiya, K, et al.. Comparison of automated SARS-CoV-2 antigen test for COVID-19 infection with quantitative RT-PCR using 313 nasopharyngeal swabs, including from seven serially followed patients. Int J Infect Dis 2020;99:397–402. https://doi.org/10.1016/j.ijid.2020.08.029.Search in Google Scholar

7. Hirotsu, Y, Maejima, M, Shibusawa, M, Amemiya, K, Nagakubo, Y, Hosaka, K, et al.. Analysis of a persistent viral shedding patient infected with SARS-CoV-2 by RT-qPCR, FilmArray Respiratory Panel v2.1, and antigen detection. J Infect Chemother 2021;27:406–9. https://doi.org/10.1016/j.jiac.2020.10.026.Search in Google Scholar

8. Hirotsu, Y, Maejima, M, Shibusawa, M, Amemiya, K, Nagakubo, Y, Hosaka, K, et al.. Prospective study of 1308 nasopharyngeal swabs from 1033 patients using the LUMIPULSE SARS-CoV-2 antigen test: comparison with RT-qPCR. Int J Infect Dis 2021;105:7–14. https://doi.org/10.1016/j.ijid.2021.02.005.Search in Google Scholar

9. Basso, D, Aita, A, Padoan, A, Cosma, C, Navaglia, F, Moz, S, et al.. Salivary SARS-CoV-2 antigen rapid detection: a prospective cohort study. Clin Chim Acta 2021;517:54–9. https://doi.org/10.1016/j.cca.2021.02.014.Search in Google Scholar

10. Wölfel, R, Corman, VM, Guggemos, W, Seilmaier, M, Zange, S, Müller, MA, et al.. Virological assessment of hospitalized patients with COVID-2019. Nature 2020;581:465–9. https://doi.org/10.1038/s41586-020-2196-x.Search in Google Scholar

11. Liotti, FM, Menchinelli, G, Marchetti, S, Morandotti, GA, Sanguinetti, M, Posteraro, B, et al.. Evaluation of three commercial assays for SARS-CoV-2 molecular detection in upper respiratory tract samples. Eur J Clin Microbiol Infect Dis 2021;40:269–77. https://doi.org/10.1007/s10096-020-04025-0.Search in Google Scholar

12. Liotti, FM, Menchinelli, G, Lalle, E, Palucci, I, Marchetti, S, Colavita, F, et al.. Performance of a novel diagnostic assay for rapid SARS-CoV-2 antigen detection in nasopharynx samples. Clin Microbiol Infect 2021;27:487–8. https://doi.org/10.1016/j.cmi.2020.09.030.Search in Google Scholar

13. Poljak, M, Korva, M, Knap Gašper, N, Fujs Komloš, K, Sagadin, M, Uršič, T, et al.. Clinical evaluation of the Cobas SARS-CoV-2 test and a diagnostic platform switch during 48 hours in the midst of the COVID-19 pandemic. J Clin Microbiol 2020;58:e00599–20. https://doi.org/10.1128/jcm.00599-20.Search in Google Scholar

14. Procop, GW, Brock, JE, Reineks, EZ, Shrestha, NK, Demkowicz, R, Cook, E, et al.. Comparison of five SARS-CoV-2 molecular assays with clinical correlations. Am J Clin Pathol 2021;155:69–78. https://doi.org/10.1093/ajcp/aqaa181.Search in Google Scholar

15. Liotti, FM, Menchinelli, G, Marchetti, S, Morandotti, GA, Sanguinetti, M, Posteraro, B, et al.. Evaluating the newly developed BioFire COVID-19 test for SARS-CoV-2 molecular detection. Clin Microbiol Infect 2020;26:1699–700. https://doi.org/10.1016/j.cmi.2020.07.026.Search in Google Scholar

16. Liotti, FM, Menchinelli, G, Marchetti, S, Posteraro, B, Landi, F, Sanguinetti, M, et al.. Assessment of SARS-CoV-2 RNA test results among patients who recovered from COVID-19 with prior negative results. JAMA Intern Med 2020:e207570. https://doi.org/10.1001/jamainternmed.2020.7570.Search in Google Scholar

17. Alexandersen, S, Chamings, A, Bhatta, TR. SARS-CoV-2 genomic and subgenomic RNAs in diagnostic samples are not an indicator of active replication. Nat Commun 2020;11:6059. https://doi.org/10.1038/s41467-020-19883-7.Search in Google Scholar

18. Rodríguez-Grande, C, Adán-Jiménez, J, Catalán, P, Alcalá, L, Estévez, A, Muñoz, P, et al.. Inference of active viral replication in cases with sustained positive RT-PCRs for SARS-CoV-2. J Clin Microbiol 2021;59:e02277–20. https://doi.org/10.1128/jcm.02277-20.Search in Google Scholar

19. Avanzato, VA, Matson, MJ, Seifert, SN, Pryce, R, Williamson, BN, Anzick, SL, et al.. Case Study: prolonged infectious SARS-CoV-2 shedding from an asymptomatic immunocompromised individual with cancer. Cell 2020;183:1901–12. https://doi.org/10.1016/j.cell.2020.10.049.Search in Google Scholar

20. van Kampen JJA, van de Vijver, DAMC, Fraaij, PLA, Haagmans, BL, Lamers, MM, Okba, N, et al.. Duration and key determinants of infectious virus shedding in hospitalized patients with coronavirus disease-2019 (COVID-19). Nat Commun 2021;12:267. https://doi.org/10.1038/s41467-020-20568-4.Search in Google Scholar

21. Dinnes, J, Deeks, JJ, Adriano, A, Berhane, S, Davenport, C, Dittrich, S, et al.. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2020;8:CD013705. https://doi.org/10.1002/14651858.cd013705.Search in Google Scholar


Supplementary Material

The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2021-0182).


Received: 2021-02-08
Accepted: 2021-03-29
Published Online: 2021-04-07
Published in Print: 2021-07-27

© 2021 Giulia Menchinelli et al., published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 29.3.2024 from https://www.degruyter.com/document/doi/10.1515/cclm-2021-0182/html
Scroll to top button