Response to: towards the rational utilization of SARS-CoV-2 serological tests in clinical practice

To the Editor,

We read with great interest the article by Padoan et al. on the analytical performance of a new immunoassay system for SARS-CoV-2 [1]. This thorough method validation demonstrated non-linearity on serial dilutions of high immunoglobulin (Ig)G and IgM pooled samples [1]. The conclusion was that the deviation from linearity occurred at levels much higher than the thresholds being used to define antibody status and was therefore not clinically significant. In our own laboratory method validation, of the Elecsys^® anti-SARS-CoV-2 assay, we also demonstrated non-linearity and we wonder if the assumption that this is of low impact may be misleading.

Samples were analysed using a Roche cobas^® e immunoassay analyser (Roche Diagnostic GmbH, Mannheim, Germany). The Roche method is an automated electrochemiluminescent assay utilising a recombinant protein representing the nucleocapsid (N) protein of SARS-CoV-2. This is a sandwich immunoassay using biotinylated SARS-CoV-2-specific recombinant antigen, SARS-CoV-2-specific recombinant antigen labelled with a ruthenium complex and streptavidin-coated microparticles (reagent lot: 50075101) [2], [3].

The output is a single result (cut off index [COI]) which is used in a qualitative fashion to determine if a sample is positive or negative for antibodies. It is not meant to represent antibody titres e.g. IgG, IgM, both or other combinations. It should be noted that all numerical values (COI) have to be reported centrally so these data are being recorded to inform national response and research. A COI<1 is considered ‘negative’ or ‘not detected’ in this assay system according to the manufacturer however some debate this [4].

Verification studies included examining the assay’s measuring range and the dilution behaviour of samples. No ethical approval was sought as samples were run anonymously for the purpose of assay verification.

Obligate COVID-19 antibody negative sera were pooled to use as a diluent. These samples were ones which had been previously analysed as part of first trimester screening. The First Trimester Screening Programme requires samples to be stored frozen for a minimum of two years; a large number of samples from 2018 were due to be discarded and were instead retained for use as negative testing material. Samples were anonymised before any further testing. The pooled sera had a COI of 0.089. COVID-19 antibody high positive samples were chosen at random and each diluted in negative pooled sera to produce individual dilution profiles, the dilutions being 1/2, 1/4, 1/8 and 1/16.

Our observations indicated that not only was extreme alinearity present but that different samples showed different dilution profiles. Overall four main dilution profiles were evident: 1) those in which some dilutions (3/4 and 1/2) gave COI values higher than the neat sample (termed “hooked”), 2) those in which some dilutions gave results very similar to the neat sample (termed “plateau”), 3) those which were almost linear and 4) those where the results from the 1/2 dilution were significantly lower than expected (these showed a sigmoid like appearance, often with multiple points of inflection) Figure 1. In most cases the shape of the full dilution profile could be predicted using just the results from analysis of the neat sample and a 1/2 dilution and so this strategy was utilised for the analysis of further samples. A total of 50 samples were analysed. Of which, 12 (24%) showed a ‘hooked’ pattern, 4 (8%) showed a ‘plateau’ pattern, 11 (22%) showed a ‘linear’ pattern and 23 (46%) showed a ‘sigmoid-like’ pattern Figure 2.

Figure 1:

Dilution curve profiles of four individual patient’s serum samples based on apparent screening pattern of neat vs. 1 in 2 dilutions (see Figure 2).

A further 3 in 4 dilution was analysed on the ‘sigmoid-like’ sample to help elucidate the dilution profile.

Figure 2:

Graphical representation of 1 in 2 dilution screening.

The top of the blue bar represents the COI of the undiluted patient sample, the bottom the expected value for the 1 in 2 dilution. The red circle is the actual measured COI of the 1 in 2 dilution.

The non-linearity therefore appears to be patient specific but we are not confident that the extent and range of patterns have been fully explored. It is also clear that one cannot predict from the COI of the neat sample the pattern found on subsequent dilution, i.e. it was independent of initial signal. However only higher COI were selected for our dilution studies and therefore lower signal samples are underrepresented. It is interesting to note that Padoan et al. saw a potential sigmoid relationship in a high patient pool sample where the impact of the starting position of each sample is limited [1].

Another hypothesis was that the patterns seen in the Elecsys^® verification could be due to different proportions of IgG and IgM in the primary sample and the dilution was not affecting each antibody to the same degree. It was of great interest therefore to see that Padoan et al. had demonstrated non-linearity for the antibodies measured separately, albeit in pooled samples [1]. In summary the conclusion that non-linearity only occurs at higher values may require further investigation and it is possible that the pattern is patient or assay specific [1].

Other groups have examined linearity for example Mariesse et al. using the iFLASH^®, demonstrated a non-linear relationship, particularly for IgG, and Tré-Hardy et al. demonstrated a plateau effect for the Liaison^® IgG assay both using a single patient sample [5], [6]. Lau et al. demonstrated a linear dilution curve on Roche but this was again a single patient sample [7]. Padoan et al. in a method comparison of five immunoassay systems have noted alinearity in all assay systems except for Roche Elecsys^® interestingly, again using a high and low patient pool, with arguably quite a high COI in the negative pool [8], [9].

The implications one could argue are limited as the current assays are not meant to be used in a quantitative fashion and quantitative assays are being introduced. However for interference studies, the use of historic data, assumptions re timing of antibody responses (decreasing and increasing titres) and derivation of thresholds for example, the effects of non-linearity are more significant [10]. In view of our results we believe that attempts to review historic data to infer changes in antibody titres over time need to be viewed with caution if using this method. In summary the work of Padoan et al. and others should be promoted as measurement uncertainty can fail to be translated and serological assays for SARS-CoV-2 are thus a limited tool to help but not significantly diminish clinical diagnostic uncertainty [1], [11].