Evaluation of five multisteroid LC‒MS/MS methods used for routine clinical analysis: comparable performance was obtained for nine analytes

Introduction

Modern laboratory diagnostics increasingly rely on liquid chromatography‒mass spectrometry (LC‒MS/MS) methods for measuring steroidal hormones. Over the past two decades, the possibilities and limitations of this technology have become well understood [1], [2], [3]. Metrological traceability has been demonstrated for several key analytes, and the IVD industry has established certified assay formats [1, 4, 5]. In addition, the in-house development of measurement methods (lab-developed tests, LDTs) is widespread in the field of endocrinology. For example, to target prepubertal estradiol and testosterone levels, the measurement range of commercially available kits must be significantly extended at its lower end; as a result, clinical needs remain unmet in certain populations (e.g., in pediatrics) [6, 7]. In addition, the sensitivity of IVD kits is often limited due to poorly optimized chromatography and mass spectrometry settings [8]. A mixed form is achieved by LDTs, which use certified and traceable commercial IVD calibration systems to minimize the bias input into the method, which is possibly associated with the production of calibrator samples.

Overall, the situation is complex, but the monitoring interlaboratory testing schemes showed a satisfactory outlook; these schemes first and foremost include the United Kingdom National External Quality Assessment Service (UK NEQAS) proficiency testing (PT) scheme for steroids, which works with real patient materials [9]. Furthermore, mass spectrometric laboratories obtain measurements with comparable precision or better than the analytical realization on the immunological high-throughput automats. However, mass spectrometric examination procedures are decisively advantageous because with these methods, an analytical design that is principally free of interference can be obtained and patient-specific systematic errors due to nonspecific measurements of the measurand in the ligand-binding assay (“cross-reactivity”) can be suppressed.

In the present study, five laboratories from Switzerland and Germany, designated Lab A–Lab E, attempted to test a central hypotheses that explains the advantage of using mass spectrometry in routine clinical diagnostics. All participating laboratories possess multianalyte LC‒MS/MS methods for diagnostic use; all laboratories are accredited according to either ISO 15189 or ISO 17025. Most of these laboratories use IVD-certified calibrator materials from one vendor (Supplementary Table S1). The aims of this study were threefold. First, the study was performed to investigate the level by which random and systematic errors contribute towards the results. Second, evaluations were performed to determine whether these figures of merit were comparable to the interlaboratory scatter found in the UK NEQAS PT scheme. As a third goal, the desirable total allowable error (TAE) derived from biological variation (BV) data was compared to the experimentally found intra- and interlaboratory errors to unveil whether these numbers exceed the TAE goals [10, 11]. The role of using IVD-certified calibrator materials is of special interest since the use of these materials may lead to reduced interlaboratory deviations [12], [13], [14], [15]. To test these hypotheses as realistically as possible, one participant purposefully prepared 40 multianalyte pooled serum samples that covered the reference interval of the individual analytes as completely as possible. In this respect, the approach differs from the recently published HarmoSter study, which focused more on targeted individual sample analyses and material comparisons [12, 13].

Materials and methods

Results

The following analytes were included in this comparison study: 17-hydroxyprogesterone, androstenedione, cortisol, and testosterone, with all five participants reporting results, and 11-deoxycortisol, corticosterone, DHEAS, cortisone and progesterone, with results available from four sites. Depending on method performance, some results were excluded from data analysis because the concentrations were below the respective locally established LOQ; one sample result was affected for testosterone and 11-deoxycortisol (Labs A and C, respectively), and eight sample results had to be excluded for DHEAS (Lab B). For progesterone, a total of 16 sample results from multiple sites were below the LOQ (Labs B–D); thus, three samples were completely excluded because the limit of three participant results per sample was not achieved. For further details, see Table 1. After the first samples were sent out to the participating laboratories, six months were needed for all samples to be processed at all sites. Sample stability over this period was assessed in Lab A and found to be sufficient (for details, see Supplementary Text S2).

The concentrations realized in the pooled samples ranged from 0.1 nmol/L for testosterone to more than 8,000 nmol/L for DHEAS. These values matched very well with biological reference intervals for healthy adults and included samples clearly below (e.g., cortisol) or above (e.g., progesterone) these cutoffs. These concentrations can be found in certain healthy subpopulations, such as children or pregnant women, or under individuals taking medications, e.g., glucocorticoid treatment. The data analysis results for these samples as the mean and range, bias and iCV data are summarized in Table 1. Bias values against the overall mean for individual samples are illustrated for each laboratory in Bland‒Altman styled plots in the Supplementary Figures S1 to S9. Generally, mean bias values were greater than ±10 % for all nine analytes at each laboratory. For androstenedione, DHEAS and progesterone, there were mean deviations from the overall mean of >10 % at one site and for cortisol and corticosterone at two sites. The maximum mean bias observed was −18.9 % at Lab C for androstenedione.

The SD of bias was constantly <10 % at all laboratories apart from Lab D, with an elevated SD of bias ranging from 8.5 to 22.3 % for all measured analytes. The SD of bias was elevated with all methods for the analytes progesterone and testosterone compared to other analytes, mainly due to higher variations at lower concentrations. The SD of bias ranged between 6.1 and 15.3 % and between 18.0 and 46.8 % for testosterone and progesterone, respectively. If the analysis is limited to samples with an overall mean of below 2 nmol/L or below 3 nmol/L for progesterone or testosterone, respectively, the SD of bias was 7.7–18.9 % for testosterone (depending on the laboratory) and 23.7–67.5 % for progesterone; for samples above those cutoffs, the SD of bias was 2.6–10.0 % and 3.5–9.7 % for testosterone and progesterone, respectively (see Supplementary Figures S10–S13).

Plots of Passing‒Bablok regression analysis conducted with the results of all laboratories are displayed in Figure 1. Detailed results from Passing‒Bablok regression analysis for intercept, slope, and the coefficient of correlation (r²) together with their respective 95 % CIs are summarized in Table 2.

Figure 1:

Passing‒Bablok regression analysis comparing single lab results with the overall mean.

Good agreement between laboratories was observed for the analytes 17-hydroxyprogesterone, cortisone and progesterone, with the 95 % CIs of the individual (Lab A–Lab E) slopes overlapping with the overall 95 % CI. For all other analytes, overlapping 95 % CI intervals were attained for all but one method per analyte (see bolded values in Table 2). The results obtained for the LC‒MS/MS methods used in Lab C and Lab B tended to be lower than those of the other participants for androstenedione, cortisol, and corticosterone and for DHEAS, respectively, while the results of Lab B and Lab D were higher on average for 11-deoxycortisol and testosterone, respectively.

Overall bias variability was compared to desirable TAE values by utilizing Bland‒Altman styled plots (Figure 2). For 17-hydroxyprogesterone, cortisol, 11-deoxycortisol, corticosterone, and DHEAS, the 1.96 SD interval of the bias values was found within TAE limits. For androstenedione and cortisone, the TAE limits closely matched the 1.96 SD interval. For cortisol, progesterone, and testosterone, a higher bias variability in samples with a mean concentration below 2 nmol/L was causal for missing the TAE targets.

Figure 2:

Bland‒Altman styled scatter plots showing the percent difference of single results of each laboratory against the overall mean (y-axis) drawn against the overall mean (x-axis). Consequently, the mean difference against the overall mean is always 0.0 %, as illustrated with a blue line. The overall 1.96 SD interval (dashed red line) is compared with the total allowable error (green line) derived from biological variation data (see Supplementary Table S2).

A comparison of iCV data from this study to iCV data from the UK NEQAS PT scheme and to TAE targets is presented in Figure 3. The iCV of most samples, including those from this study’s and the UK NEQAS sample sets, remained within desirable TAE targets (Table 1). Generally, the results for iCV were comparable between both sample sets in terms of absolute values and trends (e.g., higher iCV values at lower concentrations). The highest differences were observed in DHEAS and 17-hydroxyprogesterone. For DHEAS, iCV values were higher on average in this study than in the UK NEQAS sample set and lower for 17-hydroxyprogesterone (mean iCVs 8.1 vs. 12.7 % and 11.2 vs. 7.7 %, respectively).

Figure 3:

Comparison of interlaboratory CV values obtained from this study (white dots) and from the UK-NEQAS proficiency testing scheme (black dots) as a function of the mean sample concentration. The green line represents the optimal total allowable error (TAE), and the red line represents the desirable TAE, which was derived from the EFLM database or from the literature [8, 22], [23], [24]. LCMS group CV data from UK NEQAS PT distributions 470–494 were used, and only samples in the concentration range of the study samples were included. Study samples: all analytes, n=40; UK NEQAS samples: 17-OH-progesterone, n=56; androstenedione, n=63; cortisol, n=82; testosterone (female and male), n=150; DHEAS, n=54; progesterone, n=22.

Discussion

It was demonstrated that comparable results were obtained in most participating laboratories for the addressed analytes in terms of the mean bias against the overall mean value and the variance of biases. Most results remained clearly within the desirable TAE performance targets derived from the EFLM database biological variability data. This also includes progesterone and testosterone, even though their scatter expressed as a 1.96 SD interval in the Bland‒Altman plot clearly exceeded the TAE targets. This effect was observed only if the scatter was calculated from all samples and was assignable to poor interlaboratory agreement for very low-concentration samples. This problem may not involve general interassay agreement but rather a lack of sensitivity or specificity; in addition, low concentrations could be missed when the calibration range is too narrow for individual assays.

The study provided new insights into the quality of assay performances at all laboratories. For example, the measurement bias found at Lab C for androstenedione was not identified in the PT scheme the laboratory participated. In that particular scheme, reconstituted serum spiked with steroids was used instead of an authentic serum or pooled serum, hence an oversimplified matrix was applied. Similar problems with the commutability of external quality materials using samples with a manipulated or artificial matrix were also found in the HarmoSter study [9, 10]. However, the major reason for the deviations observed at Lab C was partly due to assay calibration issues, which was demonstrated by exchanging calibration materials with Lab C and Lab B (data not shown).

Compared to the other laboratories, Lab D exhibited problems with higher variance across the concentration range for some analytes. The possible reasons for this problem remain speculative. The pooled samples used in this study included serum from different kinds of collection tubes and from gel separator tubes. The assay in Lab D was not optimized for these materials because they are not used in routine testing at that site. The assay in Lab D also has the shortest runtime, which could limit its chromatographic resolution toward interferences from the matrix, suggesting a selectivity issue.

Furthermore, valuable information was provided for all laboratories on the individual performance of their methods in the quantification of analytes, such as 11-deoxycortisol and corticosterone; to our best knowledge, there is no existing PT scheme available using authentic materials.

The iCV data in this study were comparable to iCV values derived from the UK NEQAS PT data. This is a good indicator for the quality of the data, since UK NEQAS uses mainly pooled serum samples, although the samples can be spiked with analyte or known interferences. Differences observed between both datasets may be influenced by the low number of participants in this study or in the UK NEQAS sample set, as observed for progesterone. Since most iCV values of both data sources were within optimal TAE goals, good general agreement between LC‒MS/MS methods and sufficient performance for clinical needs can be assumed.

The influence of calibration materials could be examined in this study since two types of calibrators were used by the participating laboratories. Lab C was the only laboratory that used in-house prepared calibration materials, while the other laboratories used commercial IVD certified calibrators from one provider. The largest deviation from the overall mean was found at androstenedione for Lab C. Additionally, significant bias contributions were found at DHEAS for Lab B or at testosterone for Lab D. Therefore, the type of calibration may influence the comparability of the methods but is not the exclusive cause of bias. Differences in results between methods may also be caused by different sample preparations, chromatography settings, and different specificities against matrix interferences. Additionally, compared to the calibration type, lot-to-lot variability could exert a more significant impact since commercial and in-house calibrator materials are based on reference materials spiked into surrogate matrices. These findings also correspond with other comparison studies mentioned above.

To the best of our knowledge, this is the first published study to examine the interlaboratory performance of LC‒MS/MS measurements for DHEAS. The overall comparability of results was sufficient for clinical needs, since the overall bias variability and iCV values remained within TAE targets. However, additional variation was introduced by Lab B and Lab D, leading to lower-than-expected overall performance for DHEAS from an analytical point of view. While Lab B showed a nonlinear relative bias against the overall mean, hinting at a linearity problem, Lab D had a higher bias across the whole concentration range than that of Lab A or E. Considering that all laboratories used calibrators from the same manufacturer and that DHEAS is the most abundant steroid in the bloodstream, this result scatter is surprising.

Compared to other steroids, DHEAS exhibits an advantage in MS analysis due to its inherent properties; however, these properties could be a possible explanation for the poor analytical performance. Multisteroid methods are usually optimized for analytes that are the most challenging to measure. DHEAS, as a charged analyte, has very different physicochemical properties from those of other steroids and has a concentration approximately 10 times higher. This could lead to DHEAS measurement issues, such as mass spectrometry linearity problems due to saturation effects or unstable chromatography affecting specificity, e.g., if unbuffered mobile phases are used (e.g., ammonium fluoride in methanol/water).

There are only a few other published studies to date that compare interlaboratory LC‒MS/MS methods for steroid analytes. The published studies treat single or only a few steroidal analytes (testosterone, androstenedione, DHEA, estradiol) [27], [28], [29], [30], [31], [32], [33]. A single study, which was partly published recently (HarmoSter), dealt with a similar number of analytes as in this study [12, 13]. The HarmoSter study design is significantly different, especially the type of samples. In the study presented here, pooled serum samples were used exclusively, whereas in HarmoSter, samples were taken from individual donors with three different collection tubes each and from PT schemes. While single donor samples are the best solution in terms of commutability, the procedure we have chosen for this study is by far easier to perform (leftover-samples, sample number, sample type). Using our targeted approach in sample collection, the concentration ranges of the samples could be better controlled. Thus, we could achieve good coverage of biological reference intervals and pathological levels in a compact format of 40 samples without the need to artificially spike the samples to reach target concentrations. Hence, the commutability of these pooled serum samples should be sufficient for the purpose of this study – to evaluate the performance and comparability of routine clinical LC‒MS/MS methods.

Limitations of this study are the low number of participating laboratories and the restricted sample number, which could affect the validity of the results. However, the results of this study were substantiated by comparison to the UK NEQAS sample set and to other studies mentioned above. Furthermore, since the sample set was not value assigned by reference methods, individual measurement deviation (bias) could not be calculated against a true value but only against the overall mean.

Conclusions

This study provides a good overview that compares and describes the current state of multisteroid LC‒MS/MS-LDTs. The results provide valuable feedback to the study participants on their respective methods. In addition, generalized conclusions on the performance of LDT solutions in modern routine laboratories could be obtained. The results of this study show that, despite remarkable instrumental heterogeneity, the LC–MS/MS assays provided comparable measurement results. The result scatter was within the generally accepted performance limit (TAE) based on the BV of the endogenously present analytes. Thus, it can be concluded that the application of LC‒MS/MS for steroid analysis in clinical laboratories has developed so that an increased risk for patients from LDTs cannot be identified [34]. This observation is especially true when considering risks associated with the limitations of currently widely used fully automated immunoassays, such as cross-reactivities or limited assay sensitivities [35].

However, as for other laboratory instrumentation, constant performance monitoring of LC‒MS/MS installations in routine is necessary. This study should serve as an example of how interlaboratory quality assessment could work in addition to organized PT schemes, especially for analytes not covered by PT providers. No general decisive advantage for the comparability of measurement results could be found when commercial calibrators were utilized, but circumventing the laborious and error-prone in-house production of calibrators may increase the robustness of an assay and may facilitate traceability and standardization. Therefore, we think that the best solution for LDT-based LC‒MS/MS steroid analysis involves developing the an in-house measurement method tailored to the analyte panel needed locally and the use of standardized, traceable calibrators. This approach will enable modern personalized laboratory medicine that provides the best possible results in a flexible and timely manner as the basis of individualized decision-making by care-taking clinicians.