Abstract
Objectives
In scenarios of vaccine scarcity or contexts of organizational complexity, it is necessary to define prioritization strategies for allocating vaccine doses in compliance with the criterion of equity and efficiency of health resources. In this context, the COVIDIAGNOSTIX project, based on the health technology assessment (HTA), assessed the role of SARS-CoV-2 serological tests as a companion diagnostic in the definition of the vaccination strategies for the vaccine administration. To guarantee evidence support for health policy choices, two different vaccine strategies were analyzed, one based on administering the vaccine booster dose to the entire population (VACCINE strategy) and the other based on allocation criteria (TEST&VACCINE strategy).
Methods
The decision-oriented HTA (DoHTA) method, integrated with specific modeling and simulation techniques, helped define the perimeter to make health policy choices.
Results
The processing of the scores attributed to the key performance indicators concerning all the evaluation domains shows a performance of 94.34% for the TEST&VACCINE strategy and 83.87% for the VACCINE strategy.
Conclusions
TEST&VACCINE strategy can be the most advantageous in various scenarios due to greater speed from an operational and an economic point of view. The assessment schemes defined by COVIDIAGNOSTIX (i.e., technologies/intended use/settings) can easily and quickly be exported and adapted to respond to similar health “policy questions”.
Introduction
The BNT162b2 mRNA vaccine against SARS-CoV-2, one of the FDA-approved vaccines, has shown up to 95% protection against the worst form of COVID-19 disease, providing significant public health advantages [1]. However, previous studies of the Coronaviridae family had shown both a short-lived antibody response and the possibility of reinfection in recovered individuals [2, 3]. Therefore, as part of the ongoing Italian nationwide COVID-19 vaccination campaign, the Italian Ministry of Health has provided indications on administering an extra dose of vaccine as an additional dose (provided, as part of the primary vaccination cycle, for transplanted and immunocompromised subjects) or booster dose (provided, at least six months from the completion of the primary cycle to the general population based on the epidemiological trend).
Meanwhile, serological tests had been widely used in determining the trend of the SARS-CoV-2 antibody titer [4]; therefore, their role as a companion diagnostic was hypothesized to support the prioritization strategies of vaccination campaigns against SARS-CoV-2.
The market of serological tests is rapidly evolving; some variability was found between technological platforms, depending on the type of antigen and the technology for visualizing the antigen-antibody complex [5]. Therefore, the results obtained by one method are not comparable to those obtained by other methods [6]. Furthermore, neutralizing activity experiments [6, 7] highlight the difficulty of finding a reliable correlation between vaccine protection by assessing the antibody titers and the involvement of other immune mechanisms, such as long-lived memory B and T cells, in a protective role against SARS-CoV-2 infection.
Despite these limitations, it has been hypothesized that the SARS-CoV-2 antibody titer, integrated with other specific parameters, could be used to prioritize administering the SARS-CoV-2 vaccine. It is necessary to define prioritization strategies in vaccine scarcity scenarios or contexts of organizational complexity for allocating vaccine doses according to the criterion of equity and efficiency of health resources [6].
A valid vaccination strategy aiming to reduce severe morbidity, mortality, and harmful societal impact due to the transmission of SARS-CoV-2 allows optimization of public health interventions in facing the pandemic. The allocation criteria are focused on different risks [15]: 1) acquiring infection, 2) severe morbidity and mortality, 3) negative societal impact, and 4) transmission of the infection. Applying allocation criteria to specific population groups could establish whether a category should be prioritized in receiving the immunization, i.e., particularly for older people, whose humoral response to infections could be less efficient due to immunosenescence [8].
The main objective of the study was to analyze and compare two vaccination strategies for the administration of the booster dose, VACCINE (in which the booster dose is administered to the entire target population) and TEST&VACCINE (in which the antibody titer is used as a complementary index in defining the vaccination campaign prioritization strategy).
Materials and methods
The study was conducted through the decision-oriented health technology assessment (DoHTA) [9] (Figure 1), a health technology assessment method developed by the Bambino Gesù Children’s Hospital (OPBG) that involves a combination and integration of the HTA Core Model® for Diagnostic Technologies (version 3.0) developed by the European Health Technology Assessment Network (EUnetHTA) with a multi-criteria decision analysis (MCDA) model based on the analytic hierarchy process (AHP).
Decisional tree.
The hierarchical structure of the DoHTA method is composed of multilevel criteria: the apex is represented by health technology evaluation; the first level consists of the domains identified in the Core Model® EUnetHTA. By analyzing the Core Model assessment elements, the subsequent sub-levels consist of indicators identified through the synthesis of the evidence from the literature, the comparison with the components of the project workgroup, the results of the context analyses specific and the eligibility criteria for these indicators (evaluable a priori, applicable to all alternative technologies considered, preferably objectively measurable).
In the COVIDIAGNOSTIX project, the apex of the hierarchical structure is the strategy for administering the booster dose of the COVID-19 vaccine, and the two strategies compared are VACCINE vs. TEST&VACCINE.
The AHP allows to split the decision-making problem into evaluation sub-criteria; each criterion is given a different weight to guide the decision maker’s choice among the various alternatives. The AHP method is based on a structured questionnaire consisting of “pair comparisons” between the various evaluation elements.
The analysis covers all the evaluation domains described in the Core Model® [10] in a multistep evaluation process, as described below:
Step 1: Definition of the evaluation question
The subject of the HTA was the two alternative strategies VACCINE vs. TEST&VACCINE.
Step 2: Definition of the evaluation scheme
The complete evaluation scheme of the Core Model has been submitted to the COVIDIAGNOSTIX workgroup to evaluate the inclusion or not of the assessment elements (AE, the evaluation element defined by the combination of domain, topic, and issue). A value of 1 was assigned to the elements included and 0 to those excluded. Therefore, the AEs with a value <0.5 were excluded from the average scores attributed, while those with a value ≥0.5 were included.
Step 3: Evidence collection and literature review
The analysis of the scientific evidence took place through the consultation of PubMed and EMBASE, defining appropriate search keys. The papers identified have been managed and shared using the Zotero program (version 5.0.95), allowing the review of scientific literature according to the areas “Clinical”, “Economic and Organizational/Process”, “Technical” and “Social, Ethical and Legal” by specific COVIDIAGNOSTIX workgroup according to the following scheme: OPBG, Casa Sollievo della Sofferenza (CSS), San Raffaele Hospital (HSR), Fondazione Policlinico San Matteo (FPSM) for clinical aspects; OPBG, CSS, Orthopedic Institute Galeazzi (IOG) for organizational aspects; OPBG, IOG for economic factors; OPBG, IOG for social aspects; OPBG, CSS for ethical and legal aspects; OPBG, FPSM for technical aspects.
The research strategy is reported in the Supplementary Material, Table S1.
Step 4: Definition of key performance indicators
The Key Performance Indicators (KPIs) list was developed based on the selected Assessment Elements and information obtained from the scientific literature.
Step 5: Definition of the weighting system (priority analysis)
After identifying all the evaluation elements and the decision-making structure, the weights of the evaluation areas and each KPI were calculated. The definition of weights is a constituent part of the mathematical model of data processing, as the AHP method is based on a structured questionnaire consisting of “pair comparisons” between the various evaluation elements. Therefore, each referent by professional area was asked to compare the relative importance of the two elements on a qualitative scale (1: equal importance; 9: more important). In addition, consistency indices were computed (Consistency Index; Consistency Ratio) to check provided answers. The weights were integrated into an overall weight system through a geometric average of the weight values obtained by grouping the evaluations expressed by each interviewee.
Step 6: Attribution of performance values to KPIs
With Step 5 completed, a weighted decision tree was available for evaluating the considered alternative technologies. This was performed using alternative technologies (VACCINE vs. TEST&VACCINE) to set up another pairwise comparison list (subjected to the COVIDIAGNOSTIX workgroup). Each technology was compared against its alternative concerning every lowest indicator of the decision tree. Whatever the nature of the indicators, it is finally possible to compute (using a weighted sum) a numerical value that represents the performance of each technology concerning the overall decision goal.
To measure performances in the clinical effectiveness domain, it has been necessary to develop the SEIR (Susceptible, Exposed, Infectious, Recovered), an epidemiological model, to simulate the effects of different strategies in controlling the pandemic and preventing deaths. Then, the two vaccination strategies were evaluated by the SEIR epidemiological model (Figure 2) [11], of which the source codes are provided and executable through the free statistical software R [12]. Thanks to a specific contact matrix, the model considers the susceptibility to infection and the mortality rate from infection (IFR), which allows estimating the incidence cumulative of SARS-CoV-2 infections, mortality due to infection, and years of life lost (YLL). Furthermore, the model is stratified by age, related to susceptibility, seroprevalence, disease severity and mortality [6, 13, 14]. The model has been updated concerning the Italian epidemiological situation in October 2021 and implemented by 10 additional parameters: Italian population by age group [15]; contact matrix by age group [16]; susceptibility to COVID-19 by age group; mortality by age group; % seropositivity by age group; % vaccine efficacy by age group [17]; Rt=1.15; vaccination capacity of 200,000 vaccines/day; % vaccine rejection equal to 14% of the Italian population over 12 years; test specificity and sensitivity equal to 0.933 and 0.848 respectively [6].
Graphical representation of the used SEIR (susceptible, exposed, infectious, recovered) model.
The financial analysis was carried out to analyze the economic aspects of VACCINE vs. TEST&VACCINE, considering the real practice of the hospitals involved in the COVIDIAGNOSTIX workgroup. The serological test phases were analyzed based on interviews submitted to key opinion leaders within three hospitals. A process analysis has been developed to estimate the average cost of a serological test. Thus, every single phase has been considered, from acceptance to delivery of the report to the patient (see acceptance, collection, tube transport, instrument loading, reagent insertion, bar code reading, centrifuge, tube extraction, examination execution, validation and possible dilution, reporting, report delivery). All resources have been studied (human resources, consumables, general costs). The costs related to the laboratory equipment were not included in the total cost calculation as the various centers already had suitable environments and technologies available in their structure for carrying out the test. The amortization period for the technologies considered is over. The cost to deliver a dose of vaccine (considering, therefore, the cost of a dose and all the ancillary costs) was calculated starting from the consultation of an Italian open data, the Observatory for Covid public tenders, named Openpolis (https://bandicovid.openpolis.it/), which collected all the data relating to the costs of the vaccination campaign (thus including not only the purchasing cost of vaccine doses but also the cost of other equipment and services needed to deliver the vaccine to the population) in Italy and on which it was possible to consult a database containing the specifications of all public tenders relating to this topic.
Step 7: Integration of evidence and HTA results
The performance values for each last-level indicator are aggregated to conclude the evaluation and define the ranking of the two strategies, VACCINE vs. TEST&VACCINE.
Results
The systematic review of the scientific literature included 62 scientific articles published in international journals in the HTA study (Figure 3).
Scientific papers selection flowchart.
From the approximately 160 assessment elements of the Core Model®, 37 have been preliminarily selected. From the chosen assessment elements and the subsequent analysis of the evidence, 29 KPIs were identified. Table 1 shows the number of assessment elements and KPIs identified for each evaluation domain.
Number of assessment elements (AEs) and key performance indicators (KPIs) related to specific evaluation domains.
| Evaluation domain | AEs | KPIs |
|---|---|---|
| Health problem and current use | 6 | 4 |
| Clinical effectiveness | 10 | 7 |
| Safety | 4 | 3 |
| Costs | 5 | 3 |
| Technical characteristics | 3 | 3 |
| Organizational aspects | 7 | 10 |
| Ethical and social aspects | 2 | 2 |
| Total | 37 | 29 |
The analysis results of each evaluation domain are discussed in detail below.
Health problem and current use of the technology
The assessment elements related to the Health problem and current use of the technology domain are available in the Supplementary Material, Table S2. Vaccines played an essential role against COVID-19, as reported by Italian epidemiological data: in the period between April (date of extension of vaccination to the general population) and October 2021, the overall complete immunization is 77.8% (95% CI: 77.6–77.9%) [18]; this value measures the proportional reduction of the risk (diagnosis of infection, hospitalization, admission to intensive care and death) of observing a specific event among vaccinated subjects compared to unvaccinated ones (Figures 4 and 5). It is important to note that the hospitalization rate is eight times higher in the unvaccinated than in the whole cycle vaccinated (190.9 vs. 24.6 hospitalizations per 100,000 population); similarly, the death rate is 13 times higher in the unvaccinated than in the whole cycle vaccinated (126.5 vs. 9.8 per 100,000 inhabitants). Surprisingly, in the over 80 age group, in which vaccination coverage is higher than 90%, most hospitalizations relate to vaccinated subjects; this phenomenon can be explained by the so-called paradox effect that occurs when vaccinations in the population reach high levels of coverage. However, it remains understood that incidences (diagnosis, hospitalization, Intensive Therapies, deaths) in unvaccinated populations remain 5–12 times higher than in people with a completed vaccine cycle.
Trend of Covid-19 cases diagnosed, hospitalized, admitted to intensive care, and died in different age groups in unvaccinated subjects e in subjects with a complete or incomplete vaccination cycle.
Distribution by vaccination status and age groups of Covid-19 cases diagnosed, hospitalized, hospitalized in intensive care, and dead.
Clinical effectiveness
The assessment elements related to the Clinical effectiveness domain are available in the Supplementary Material, Table S3. Both indicators relating to the outcome of the two strategies being evaluated (VACCINE vs. TEST&VACCINE) and indicators relating to the diagnostic capacity of the technologies were used.
The outcome of the two vaccination strategies, evaluated by the SEIR epidemiological model, is shown in Figure 6. The higher reduction of infections (Figure 6A) would be obtained by implementing a vaccination strategy that prioritizes the younger and, therefore, more socially active population (green line). Instead, a reduction in infections of about 3% would be obtained by a vaccination campaign that prioritizes the population over 60. With the TEST&VACCINE strategy, a decrease in infections of about 4.5% would be obtained.
SEIR model and simulation results for VACCINE (continuous line) and TEST&VACCINE strategies (dotted line) for each target population (age group) for which the vaccination campaign is to be prioritized.
Considering the distribution of vaccines set at a distribution capacity of 200,000 daily doses (equal to 0.331% of the Italian population) for the total capacity (“total vaccine supply − % total population”), the epidemiological evolution is represented respectively by the reduction of infections (A), deaths (B) and years of life lost (C). The curves appear flattened in the range 0–15%; this could be due to the Rt value set equal to 1.15. The black dot represents the immunization of 70% of the target population (% of the vaccinated population beyond which the need for population prioritization disappears), an objective after which the synthetic model simulates the start of vaccination of the entire population.
The higher reduction in mortality (Figure 6B) would be obtained by prioritizing the age group of the over 60s (purple line) and would be approximately 8% with the VACCINE strategy and 11% with the TEST&VACCINE strategy. By prioritizing this same age group, a 7.5% reduction in years of life lost is also obtained for the vaccination-only strategy, up to 10.5% in the case of TEST&VACCINE.
The objective of administering vaccination to 70% of the “actual” target population, on the other hand, is achieved by consuming about 64% of the doses that will be available in Italy by 12/31/21.
(given by the sum of the reserve of approximately 11.7 million remaining at 10/31/21 and subsequent deliveries of approximately 8.6 M expected within the year) for the vaccination-only strategy, against 48% of the TEST&VACCINE strategy. Furthermore, the reduction of the doses to be delivered allows to speed up the vaccination campaign, thus allowing the achievement of the vaccination target (that corresponds to the maximum decrease in deaths and infections) in fewer days for the TEST&VACCINE strategy than the VACCINE strategy (35÷55 days vs. 60÷90 days, at equal supply capacity).
The “number of days off work/school” indicator considers the days of absence from work or school attributable to COVID-19. The concept of reducing sick days, usually linked to abstention from work, has also been extended to school activities since COVID-19 is a disease with an extensive social impact on all age groups. For the evaluation of this indicator, the Relative Risk (RR) was calculated, focusing on the reduction of infections in two scenarios characterized by an Rt equal to 1.15 (Italian Rt at the end of October 2021) and equal to 1.57 (Israeli Rt of June 2021, the month in which the Israeli government decided to start the vaccination campaign for the booster dose). In this way, comparing the two strategies (TEST&VACCINE vs. VACCINE), we obtain variable RR values from 0.98 (for the Italy case) and 0.90 (for the Israel case) as a function of the Rt. While a value of RR close to 1 does not allow to attribute a higher performance value to one strategy rather than the other, considering the dependence between RR and Rt: in conditions of increased Rt (a circumstance that is occurring in Italy, starting from the end of October 2021) significantly better results are obtained, in terms of RR, by adopting a TEST&VACCINE strategy.
Safety
The assessment elements related to the Safety domain are available in the Supplementary Material, Table S4. The safety aspects are represented by the intrinsic safety indicator of the technology as the evaluation of adverse events. From the database of the Ministry of Health of Israel (MOH, https://data.gov.il/dataset/covid-19), it was possible to compare the number of adverse cardiovascular events (see myocarditis) resulting in hospitalization or death after the first, second, third dose in the vaccinated population compared to the unvaccinated one in the various age groups of the people. The above data shows that adverse events after booster doses are lower than post-second doses [19, 20]. If this incidence value were applied to Italy, where the target population (over 60, healthcare workers and the frail under 60) is approximately 18097344.67, a TEST&VACCINE strategy would reduce the overall absolute number of adverse events compared to the VACCINE strategy. The evaluation of the antibody titer and the possible postponement of the vaccination would reduce the number of subjects vaccinated. Consequently, the booster doses to be administered would be lower. Finally, the impact on the environment must also be considered. In fact, in the hypothesis of pursuing the TEST&VACCINE strategy, serological tests would certainly grow substantially, resulting in an increase in hazardous medical waste at infectious risk and especially of reagents used for the analysis.
Costs and economic evaluation
The assessment elements included in the Costs and economic evaluation domain are available in the Supplementary Material, Table S5.
The average cost of a serological test was estimated equal to 13.12€. The total cost of the vaccination campaign is estimated at 3.51 billion Euros (3,507,256,922). Therefore, dividing this value by the total vaccine doses administered in Italy (up to 12/10/2021) equal to 86,665,355, it is estimated that a unit dose of vaccine costs about €40.47.
The two vaccination strategies in question were compared to determine the most efficient economic resources to be used, with equal effectiveness in containing the pandemic. The financial analysis was conducted by initially including in the vaccination campaign for the administration of the booster dose a target population (priority) composed of subjects over 60, health and social health personnel (under 60) and frail subjects (under 60). The cost of the vaccination campaign relating to the entire target population without carrying out the serological test, thus excluding the possibility of delaying or excluding any subject, is equal to €732,399,539. The cost of the TEST&VACCINE strategy was calculated assuming that, following the serological test, a subset (%) of the target population (18,097,345) to be vaccinated could be identified as a priority. In contrast, the booster dose can be excluded or postponed for the remaining part of the population. The total cost of the TEST&VACCINE strategy is calculated as the sum of the expected cost of vaccinating the subset of the target population (above) and the total cost of testing the entire target population (€237,437,162).
Following the analysis previously described the subset of the target population for which the economic equivalence of the two vaccination strategies occurs was calculated:
The percentage of 67.6% represents the subset of the target population to be vaccinated after carrying out the test, making the two vaccination campaigns equivalent in public health expenditure. Therefore, from an economic point of view, for values lower than this percentage, implementing the TEST&VACCINE strategy would lead to minor costs compared with the VACCINE strategy. On the contrary, the latter would lower costs than the TEST&VACCINE strategy for values above this threshold. In other words, following the serological test’s execution to the entire target population, it is necessary to administer the third vaccination dose to a percentage of subjects lower than 67.6% of the tested ones; the TEST&VACCINE strategy would lead to minor costs compared with the VACCINE strategy. This could impact the financial planning of the National Health Service, and, in case of low Rt values (Rt<<1), it might lead to avoiding the administration of booster doses, which were initially postponed.
Considering the National Health Service’s perspective, different scenarios were evaluated based on the SEIR model to estimate the “Clinical efficacy” indicators to analyze the budget impact. A simulation was carried out with the model reported above regarding the previously described economic equivalence (see parameter % Seropositivity by age group equal to 67.6%), whose results show that the TEST&VACCINE strategy should be preferred from the point of view of the economic resources necessary to vaccinate 70% of the target population, leading to minor costs compared with the VACCINE strategy. In fact, with the TEST&VACCINE solution, the optimization model expects to dispense 48% of the supply (a total of 20.3 million doses available by 12/31/21) against about 64% of the VACCINE strategy to achieve the maximum reduction of dead and YLL. There would be a saving (Figure 7) of approximately 3.2 million doses to be delivered to achieve the desired vaccination coverage. Consequently, almost 131 million euros would be lower costs (number of doses dispensed less * vaccine dose unit cost). The data shown highlights the different impacts the strategies would have on the National Health Service budget.
Comparison of TEST&VACCINE and VACCINE strategies (savings of about 3.2 million doses to be delivered to achieve the desired vaccination coverage and consequently lower costs by about €131 million).
Cost-effectiveness analysis can help identify gaps in evidence on the estimated effects of public health interventions and support the decision to disinvest older interventions with more cost-effective alternatives. The cost-effectiveness analysis provides a reference framework for comparing the value of the different interventions; it can assist decision-makers in classifying other options and define related efficiency in resource allocation, providing helpful information for their programmatic and financial needs.
In the case of the two strategies TEST&VACCINE and VACCINE, it was not possible to calculate the cost-effectiveness ratio of the two strategies evaluated due to insufficient data available; therefore, the cost-effectiveness analysis was carried out by conducting simulations using the SEIR model and adopting a cost-consequences approach. In terms of effectiveness, reducing deaths (%) can be considered as previously described in Figure 6B. The TEST&VACCINE strategy determines a significant reduction in deaths compared to VACCINE, especially for the over 60 age group (see dotted vs. solid lines in purple).
As for costs, as previously described, with the TEST&VACCINE strategy, it would be possible to vaccinate 70% of the target population with only 48% of the expected supply in Italy as of 31/12/21, while the VACCINE strategy requires almost 64% of the supply.
Therefore, the TEST&VACCINE scenario shows higher efficacy (significant reduction in deaths) at a lower cost (a significant number of vaccine doses saved) than the VACCINE strategy, suggesting potential domination to be confirmed in future with the implementation of long-term cost-effectiveness models.
Description and technical characteristics of technology
The assessment elements included in the Description and technical characteristics of the technology domain are available in the Supplementary Material, Table S6. The peculiar features of the multiple tests (type of antibody detected, target antigen, the method used, analytical turn-around time (TAT), analyzer productivity) were previously investigated in phase I of the project [5] and summarized in Table 2. The indicators refer to the need for dedicated places to execute the two investigated strategies. In particular, the difference found between the two strategies is the booster’s need for dedicated spaces for carrying out the test and analyzing the samples in the TEST&VACCINE case. However, this limit could be exceeded, thus considering the two strategies as equivalent, assuming that the serological tests are carried out at the facilities already in place, which is why, in this regard, there would be no additional costs even for the TEST&VACCINE strategy. Moreover, the TEST&VACCINE strategy involves the use of additional materials for the execution of the sampling and for the implementation of the analytical test (gloves, butterfly needle, vacutainer, blood containers, disinfectant, gauze, cotton, labels, soap or disinfectant gels, reagents, etc.) for which it is necessary to consider them as an additional burden to be considered in the evaluation and comparison of strategies.
Technical characteristics of the serological tests included in the study.
| Developer | Test | Technology | Target | Antigen | Throughput | Time to result |
|---|---|---|---|---|---|---|
| Roche Diagnostics | Elecsys AntiSARS-CoV-2 S | High throughput | Pan-Ig | Spike (RBD) | Up to 170 tests/h on COBAS 601 | 18 min |
| ECLIA (qualitative and semiquantitative) | ||||||
| DiaSorin | LIAISON SARS-CoV-2 S1/S2 IgG | High throughput | IgG | Spike (S1–S2) | Up to 170 tests/h on LIAISON XL | 35 min |
| CMIA (qualitative) | ||||||
| LIAISON SARS-CoV-2 Trimerics IgG | High throughput | IgG | Spike | 171 tests/hour on LIAISON XL | 35 min | |
| CLIA (qualitative and semiquantitative) | ||||||
| Siemens Healthcare Diagnostics Inc. | ADVIA Centaur | High throughput | IgG | Spike (S1 RBD) | Up to 240 tests/h on ADVIA Centaur XP/XPT | 18 min |
| SARS-CoV-2 | CLIA (qualitative and semiquantitative) | |||||
| IgG (sCOVG) | ||||||
| Euroimmun | SARS-COV-2 | ELISA (qualitative) | IgG | Spike (S1) | Up to 77 tests/h on EUROIMMUN Analyzer I | N/A |
| ELISA (IgG) | ||||||
| Diesse | CHORUS SARS-CoV-2 “NEUTRALIZING” Ab | Immunoenzymatic method (quantitative) | Pan-Ig | Spike (S1) | N/A | N/A |
| Abbott | AdviseDx SARS-CoV-2 IgG II | Semiquantitative high throughput CMIA | IgG | Spike | Up to 200 tests/h on Alinity i | N/A |
Organizational aspects
The assessment elements included in the Organizational aspects domain are available in the Supplementary Material, Table S7. The analysis of the process for performing serological tests (Health delivery process), already carried out in the first phase of the COVIDIAGNOSTIX project [5], considers all the steps from blood sampling to delivery of the report to the patient. The performance indicators identified refer to specific characteristics of the technologies which, together with the purely clinical ones (see the method of analysis, detected antigen, sensitivity and specificity of the test), can lead to their choice. For example, an analyzer with higher productivity (number of tests per hour that can be carried out) and with a turn-around time (TAT, that is, the time between the test request and the moment in which the result returns to the applicant) lower is undoubted to be preferred in a pandemic situation with a high rate of infection and deaths as it is necessary to test a larger quantity of the population and as quickly as possible. At the same time, this aspect can be considered less relevant if the test is used in the pandemic’s control/surveillance phase. The different degree of automation of the instrumental platforms used to analyze the sample also affects the test delivery process.
Conversely, laboratory personnel are independent of the test used. All these considerations are also relevant for the comparison of the two strategies. In fact, in the TEST&VACCINE strategy, it is necessary to carry out different logistics, personnel, and costs than the VACCINE strategy. Still, with adequate investments, the organizational difficulty of the first strategy can be managed and, indeed, could lead to economic savings (see Economic evaluation and analysis). In this phase of the project, wanting to evaluate the efficiency of the two strategies, without considering how other analyzers impact time and costs of the whole process, the performance of the two strategies for these elements was considered identical (always in the hypothesis of a perfect analyzer imposed in the simulation model).
The possibility of decentralization is different between the two strategies. In TEST&VACCINE, the booster dose of vaccine is administered based on the serological test result. It is essential to underline that the serological test must be performed in specialized laboratories and not through the point of care (POCT) currently available on the market; although POCTs ideally involve a saving of resources and time, they are not presently equivalent to classical serological tests either in terms of quantitative determination or in terms of specificity, sensitivity, reliability, and accuracy of the results.
To date, the vaccine has been administered not only in the central hubs but also using and adapting ad hoc different structures in the area (for example, general practitioner [GP]’s offices, gyms, schools, vaccination centers, tents in stations/airports, vaccination centers set up ad hoc, etc.).
Therefore, the VACCINE strategy for this indicator has been attributed to higher performance than the TEST&VACCINE strategy. Additionally, initial implementation costs typically affect whether introduce new technology into an organization. However, in the case of COVIDIAGNOSTIX, the centers belonging to the project network already had the necessary equipment available to analyze the serological tests and, therefore, did not incur any initial costs. In general, at present (considering the ability to perform serological tests guaranteed by the NHS structures to be sufficient), there are no problems with carrying out a serological test, just as it is not particularly difficult to receive a dose of vaccine. Therefore, the performance attributed to the two strategies for this indicator is equal.
In addition, regarding the stability of intra/inter-laboratory results, the first international standard (WHO-IS) produced by WHO was purchased and used within COVIDIAGNOSTIX to facilitate the standardization and comparison of serological methods.
Finally, the acceptability of the booster dose after serological testing was estimated based on the data [21] relating to the results of five surveys conducted on the American territory on the population already vaccinated with a double dose. In the surveys, subjects were asked about their propensity to receive a booster dose of the vaccine. About two-thirds of vaccinated adults said they would accept a possible booster dose, prioritizing the elderly and health professionals. The surveys show that respondents ready to take a booster dose are 76 and 87%. In one of the surveys, this percentage rose to 93% as the same population was asked what their propensity was to undergo the third dose following a recommendation from their GP. If a GP can suggest to his client the booster dose based on the results of a serological test, this data was considered helpful for estimating the cultural advantage of the two compared strategies, not having other data and methods to quantify this indicator. In addition, in support of this result, citizens increased out-of-pocket spending recently to perform a serological test to assess their antibody level a few months after the double dose vaccination cycle.
Therefore, in evaluating the performance of the two strategies relating to the acceptability of the third dose, a value higher than the TEST&VACCINE one was attributed based on the above data.
Ethical analysis, patients and social aspects
The assessment elements included in the Ethical analysis, patients and social aspects domain are available in the Supplementary Material, Table S8. Antibody tests for SARS-CoV-2 were essential supporting tools for monitoring incidence and prevalence trends, serum surveillance, and estimating the trend of the antibody titer. In the survey conducted in November 2020 by Nehme et al., when asked why serological tests might be helpful, about 80% agreed that the presence of antibodies would encourage people to resume activities they had abandoned due to a pandemic. At the same time, 76% of subjects agreed that the absence of antibodies would encourage people to wear masks (80.8% of individuals aged 65 and over, 76.2% of individuals aged 40–64 and 69% of individuals under 40 years of age; p=0.01).
Similarly, 87.4% of subjects agreed that the absence of antibodies would encourage people to respect social distancing measures (89.3% in individuals 65 years and older, 88.7% in individuals 40–64 years and 82% in subjects less than 40 years old; p=0.01). When asked about the perceived risks of the Green Pass, 67.7% of participants agreed that discrimination against those without immunity is a risk. Considering the difficulties in finding valuable data for evaluating the indicator’s performance above for the two strategies in question, this KPI was not included in the evaluation scheme.
In Italy, the serological tests are carried out under the solvency regime by the citizen. They are not reimbursable as they are not part of the benefits paid by the national health system. Reimbursement of out-of-pocket expenses incurred in carrying out a serological test can only be obtained from certain health insurers. In this case, the citizen can carry out the test in affiliated health facilities authorized by their insurance, and the latter will bear the cost of the examination carried out; otherwise, if the test is performed in non-affiliated facilities, it will then be possible to proceed with the request for reimbursement.
The complete list of indicators deriving from each study phase and relating to specific evaluation dimensions is shown in Table 3.
Summary of the TEST&VACCINE vs. VACCINE strategies HTA for administering the booster dose of the vaccine for COVID-19 (Evaluation areas, Lev1-KPI and Lev2-KPI).
| Domain, Lev1-KPI | Lev2-KPI | Weights, % | TEST&VACCINE performance, % | VACCINE performance, % |
|---|---|---|---|---|
| Clinical effectiveness | 23.32 | 23.32 | 20.49 | |
|
|
||||
| Mortality and morbidity | 6.79 | |||
| Speed of vaccination coverage | 6.79 | 6.79 | 5.28 | |
| Function | 5.41 | |||
| Days of infection | 5.41 | 5.41 | 4.81 | |
| Test accuracy | 4.57 | |||
| Sensitivity and specificity | 1.96 | 1.96 | 1.96 | |
| Manufacturers performance | 1.12 | 1.12 | 1.12 | |
| Interoperability and intraoperability | 1.48 | 1.48 | 1.48 | |
| Change-in management | 6.56 | |||
| Optimizing allocation for vaccines | 6.56 | 6.56 | 5.83 | |
|
|
||||
| Safety | 19.98 | 18.59 | 13.22 | |
|
|
||||
| Patient safety | 12.18 | |||
| Adverse events post third dose | 12.18 | 12.18 | 5.41 | |
| Risk management | 3.13 | |||
| Third dose registry tracking | 3.13 | 2.79 | 3.13 | |
| Environmental safety | 4.67 | |||
| Disposal of reagents | 4.67 | 3.63 | 4.67 | |
|
|
||||
| Costs and economic evaluation | 14.55 | 14.55 | 12.40 | |
|
|
||||
| Resource utilization | 14.55 | |||
| Resources identification and quantification | 5.00 | 5.00 | 3.93 | |
| Budget impact | 3.85 | 3.85 | 3.41 | |
| Cost effectiveness | 5.70 | 5.70 | 5.06 | |
|
|
||||
| Description and technical characteristics of technology | 12.13 | 11.75 | 12.13 | |
|
|
||||
| Features of the technology | 5.30 | |||
| Specific features | 5.30 | 5.30 | 5.30 | |
| Investments required | 6.82 | |||
| Installation requirements/structural modifications | 3.41 | 3.41 | 3.41 | |
| Consumables required | 3.41 | 3.03 | 3.41 | |
|
|
||||
| Organizational aspects | 16.56 | 12.67 | 15.16 | |
|
|
||||
| Health delivery process | 3.03 | |||
| Turn around time (TAT) | 0.59 | 0.59 | 0.59 | |
| Test productivity | 0.59 | 0.59 | 0.59 | |
| Professionals involved in the process | 0.41 | 0.41 | 0.41 | |
| Degree of automation | 0.85 | 0.85 | 0.85 | |
| Quality controls | 0.59 | 0.59 | 0.59 | |
| Structure of health care system | 3.03 | |||
| Decentralization | 3.03 | 1.01 | 3.03 | |
| Management | 4.20 | |||
| Inter- and intra- laboratory stability | 4.20 | 2.33 | 4.20 | |
| Culture | 6.30 | |||
| Public acceptance of the third dose | 6.30 | 6.30 | 4.90 | |
|
|
||||
| Ethical analysis, patients and social aspects | 13.46 | 13.46 | 10.47 | |
|
|
||||
| Justice and equity | 13.46 | |||
| Cost opportunity of allocating human and economic resources | 13.46 | 13.46 | 10.47 | |
-
Table shows the complete list of indicators from each study phase and relates to specific evaluation dimensions with the performance values and absolute weights.
After identifying the indicators, the COVIDIAGNOSTIX team assigned performance/score values to each indicator. Each professional, representing the institutions involved and each aspect of their competence, analyzed the specific indicator and expressed an assessment based on the evidence and data collected. The overall weighting system relating to the evaluation areas is shown in Figure 8.
Weight system related to the evaluated domains (or dimensions or domains).
Finally, the processing of the scores attributed to the KPIs concerning the relative weights shows 94.34% for the TEST&VACCINE strategy and 83.87% for the VACCINE strategy (Figure 9).
Performance values of the TEST&VACCINE vs. VACCINE strategies for administering the booster dose of the COVID-19 vaccine.
Discussion
The severe acute respiratory syndrome from SARS-CoV-2 has threatened the world population’s health. In fact, in March 2020, the Director-General of the WHO declared the “pandemic” state. The spread of a wide range of medical technologies (see vaccination, serological and diagnostic tests, etc.) has significantly impacted the “health conditions” of the population.
The COVIDIAGNOSTIX HTA project, conducted with the DoHTA method by a large multidisciplinary Working Group, coordinated by the OPBG HTA Service, compared the VACCINE vs. TEST&VACCINE strategies for administering the booster dose of the COVID-19 vaccine, showing a superior performance for the TEST&VACCINE strategy (94.34 vs. 83.87%).
A proper prioritization strategy should consider numerous variables such as age, gender, clinical history, and serostatus. In different viral infections, serological evaluation plays a crucial role in identifying unprotected individuals, distinguishing primary disease from reactivation, and testing the immune response elicited by vaccination (e.g., hepatitis B virus and Varicella-Zoster virus) [22, 23]. The antibody titer dosage could be an index of the immune response, informing about immunization coverage and durability against a specific pathogen.
A complex decision-making problem is typically characterized by a multiplicity of aspects, points of view or even decision-makers that prevent it from being synthesized into a single coherent objective. In such situations, using multi-criteria analysis models (MCAM) allows schematizing the problem in different sub-objectives, comparing and ordering the alternative solutions considered, and integrating the results of each comparison. This way, the decision-maker is provided with a structured and consistent document to identify an acceptable compromise between the various objectives. The multiple objectives of a general nature must first be transformed into criteria based on which the multiple alternatives can be compared. Furthermore, it is possible to attribute a different weight to each criterion identified for evaluation; the weight is proportional to its contribution to achieving the general objective. The attribution of the weights to the other criteria can be carried out according to various schemes proposed in the literature, among which the Analytic Hierarchy Process (AHP) method has been selected for its ease of use as well as for its versatile and robust mathematical structure. Since 2009, the HTA team of the Bambino Gesù Children’s Hospital has developed the DoHTA (Decision-oriented Health Technology Assessment) method [9, 24] that integrates AHP into the Core Model® (EUnetHTA), which was the method used in COVIDIAGNOSTIX.
The HTA of the vaccine strategies allows supporting effectively, through the data and information produced and synthesized through the presented operational tool, a thoughtful decision-making process of Health Policy. The possibility of defining different temporal priorities in the target population would constitute an essential element for optimizing the vaccination campaign, reducing the number of deaths, and containing the pandemic. This strategy of expanding over time the number of subjects to be subjected to a booster dose would also guarantee a reduction in the number of adverse events to be managed in specific time windows, being potentially able to determine an overall decrease in the number of doses delivered and consequently an absolute reduction in adverse events. From the analyses conducted, it also emerges that the TEST&VACCINE strategy can result in numerous scenarios as the most advantageous not only from an economic point of view but above all from an operational point of view, allowing to faster reach the vaccine target, with the same efficacy.
A crucial element in COVIDIAGNOSTIX is citizens’ greater acceptability and trust towards a TEST&VACCINE strategy. Moreover, the relevance of this issue for vaccinated Italians is demonstrated by the huge out-of-pocket expenditure recorded in recent months, precisely for the execution of pre-and post-vaccination serological tests. Adopting and communicating a verification process (through the test) of the immune status and subsequently indicating a vaccination calendar, in addition to determining greater acceptability by citizens, could constitute a factor of rapprochement between them and the political class and renewal of trust in the scientific community.
The TEST&VACCINE strategy is associated with higher performance in the economic sphere, resulting in a significant reduction of resources on the income statements. This solution makes it possible to make both direct costs (personnel, equipment, materials, etc.) and indirect costs more efficient (for example, in terms of time expressed in fewer days needed to reach vaccination coverage with the same delivery capacity) and consequently has a positive impact on the organizational sphere.
Conclusions
The applied DoHTA method (given by the combination of the EUnetHTA Core Model and AHP), completed with appropriate modeling techniques, simulation, and quantification of uncertainty (also to use primary data from ongoing trials), proved to be particularly useful in defining the perimeter within which to make health policy choices, proving suitable, even in the circumstances characterized by great uncertainty and the need to act promptly, to guarantee evidence support for health policy choices. However, all the considerations elaborated must consider the initial hypothesis imposed in the SEIR model. It has been hypothesized that, apart from the instrumental error, the ability to rule in and out of the model’s single-subject simulated is substantially perfect. This represents a limitation of the model used in the transferability of the results in other contexts that can be overcome by creating a classification system of subjects closest to the simulated ideal.
Finally, the assessment schemes defined in the COVIDIAGNOSTIX project can easily and quickly be exported and adapted to respond to similar health policy questions.
Acknowledgments
The Authors thank the members of the working group: Sami Al Bitar Nehme, Elena Bassanelli, Elena Cittera, Ilaria Cristiano, Liliana De Vivo, Davide Ferrari, Francesco Giuffrida, Anna Lo Russo, Selenia Marino, Stella Pastore, Michela Pireddu, Francesco Ricciardi and Sara Zacchetti.
-
Research funding: The study was granted by the COVID-2020-12371619 call funded by the Italian Ministry of Health.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Competing interests: Authors state no conflict of interest.
-
Informed consent: Informed consent was obtained from all individuals included in this study.
-
Ethical approval: The local Institutional Review Board deemed the study exempt from review.
References
1. Polack, FP, Thomas, SJ, Kitchin, N, Absalon, J, Gurtman, A, Lockhart, S, et al.. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med 2020;383:2603–15. https://doi.org/10.1056/nejmoa2034577.Search in Google Scholar
2. Chia, WN, Zhu, F, Ong, SWX, Young, BE, Fong, SW, Le Bert, N, et al.. Dynamics of SARS-CoV-2 neutralising antibody responses and duration of immunity: a longitudinal study [published correction appears in Lancet Microbe. 2021;2:e179. Lancet Microbe 2021;2:e240–9. https://doi.org/10.1016/s2666-5247(21)00025-2.Search in Google Scholar
3. Huang, AT, Garcia-Carreras, B, Hitchings, MDT, Yang, B, Katzelnick, LC, Rattigan, SM, et al.. A systematic review of antibody-mediated immunity to coronaviruses: kinetics, correlates of protection, and association with severity. Nat Commun 2020;11:4704. https://doi.org/10.1038/s41467-020-18450-4.Search in Google Scholar PubMed PubMed Central
4. Pouquet, M, Decarreaux, D, Prévot-Monsacré, P, Hervé, C, Werner, A, Grosgogeat, B, et al.. Nationwide seroprevalence of SARS-CoV-2 IgG antibodies among four groups of primary health-care workers and their household contacts 6 months after the initiation of the COVID-19 vaccination campaign in France: SeroPRIM study protocol. Pathogens 2021;10:911. https://doi.org/10.3390/pathogens10070911.Search in Google Scholar PubMed PubMed Central
5. Tomaiuolo, R, Derrico, P, Ritrovato, M, Locatelli, M, Milella, F, Restelli, U, et al.. COVIDIAGNOSTIX: health technology assessment of serological tests for SARS-CoV-2 infection. Int J Technol Assess Health Care 2021;37:e87. https://doi.org/10.1017/s0266462321000441.Search in Google Scholar PubMed
6. Ferrari, D, Clementi, N, Spanò, SM, Albitar-Nehme, S, Ranno, S, Colombini, A, et al.. Harmonization of six quantitative SARS-CoV-2 serological assays using sera of vaccinated subjects. Clin Chim Acta 2021;522:144–51. https://doi.org/10.1016/j.cca.2021.08.024.Search in Google Scholar PubMed PubMed Central
7. Ferrari, D, Clementi, N, Criscuolo, E, Ambrosi, A, Corea, F, Di Resta, C, et al.. Antibody titer kinetics and SARS-CoV-2 infections six months after administration with the BNT162b2 vaccine. Vaccines 2021;9:1357. https://doi.org/10.3390/vaccines9111357.Search in Google Scholar PubMed PubMed Central
8. Ray, D, Yung, R. Immune senescence, epigenetics and autoimmunity. Clin Immunol 2018;196:59–63. https://doi.org/10.1016/j.clim.2018.04.002.Search in Google Scholar PubMed PubMed Central
9. Ritrovato, M, Faggiano, FC, Tedesco, G, Derrico, P. Decision-oriented health technology assessment: one step forward in supporting the decision-making process in hospitals. Value Health 2015;18:505–11. https://doi.org/10.1016/j.jval.2015.02.002.Search in Google Scholar PubMed
10. EUnetHTA (European Network for Health Technology Assessment) Core Model® version 3.Search in Google Scholar
11. Bubar, KM, Reinholt, K, Kissler, SM, Lipsitch, M, Cobey, S, Grad, YH, et al.. Model-informed COVID-19 vaccine prioritization strategies by age and serostatus. Science 2021;371:916–21. https://doi.org/10.1126/science.abe6959.Search in Google Scholar PubMed PubMed Central
12. r-project.org. Available from: www.r-project.org [Accessed 9 May 2022].Search in Google Scholar
13. Di Resta, C, Ferrari, D, Viganò, M, Moro, M, Sabetta, E, Minerva, M, et al.. The gender impact assessment among healthcare workers in the SARS-CoV-2 vaccination-an analysis of serological response and side effects. Vaccines 2021;9:522. https://doi.org/10.3390/vaccines9050522.Search in Google Scholar PubMed PubMed Central
14. Ferrari, D, Di Resta, C, Tomaiuolo, R, Sabetta, E, Pontillo, M, Motta, A, et al.. Long-term antibody persistence and exceptional vaccination response on previously SARS-CoV-2 infected subjects. Vaccine 2021;39:4256–60. https://doi.org/10.1016/j.vaccine.2021.06.020.Search in Google Scholar PubMed PubMed Central
15. WHO 2019 revision of world population prospects. Available from: https://population.un.org/wpp/ [Accessed 9 May 2022].Search in Google Scholar
16. Prem, K, Zandvoort, KV, Klepac, P, Eggo, RM, Davies, NG, Cook, AR, et al.. Projecting contact matrices in 177 geographical regions: an update and comparison with empirical data for the COVID-19 era. PLoS Comput Biol 2021;17:e1009098. https://doi.org/10.1371/journal.pcbi.1009098.Search in Google Scholar PubMed PubMed Central
17. Task force COVID-19 del Dipartimento Malattie Infettive e Servizio di Informatica, Istituto Superiore di Sanità. Epidemia COVID-19. Aggiornamento nazionale: 13 ottobre 2021. Available from: www.epicentro.iss.it/coronavirus/bollettino/Bollettino-sorveglianza-integrata-COVID-19_13-ottobre-2021.pdf.Search in Google Scholar
18. governo.it. Available from: https://www.governo.it/it/cscovid19/report-vaccini/ [Accessed 18 Mar 2022].Search in Google Scholar
19. Witberg, G, Barda, N, Hoss, S, Richter, I, Wiessman, M, Aviv, Y, et al.. Myocarditis after Covid-19 vaccination in a large health care organization. N Engl J Med 2021;385:2132–9. https://doi.org/10.1056/nejmoa2110737.Search in Google Scholar
20. Mevorach, D, Anis, E, Cedar, N, Bromberg, M, Haas, EJ, Nadir, E, et al.. Myocarditis after BNT162b2 mRNA vaccine against Covid-19 in Israel. N Engl J Med 2021;385:2140–9. https://doi.org/10.1056/nejmoa2109730.Search in Google Scholar PubMed PubMed Central
21. Evidence to recommendation framework: Pfizer-BioNTech COVID-19 booster dose. Available from: https://www.cdc.gov/vaccines/acip/meetings/downloads/slides-2021-9-23/03-COVID-Oliver.pdf [Accessed 9 May 2022].Search in Google Scholar
22. Poovorawan, Y, Chongsrisawat, V, Theamboonlers, A, Leroux-Roels, G, Crasta, PD, Hardt, K. Persistence and immune memory to hepatitis B vaccine 20 years after primary vaccination of Thai infants, born to HBsAg and HBeAg positive mothers. Hum Vaccin Immunother 2012;8:896–904. https://doi.org/10.4161/hv.19989.Search in Google Scholar PubMed PubMed Central
23. Freer, G, Pistello, M. Varicella-zoster virus infection: natural history, clinical manifestations, immunity and current and future vaccination strategies. New Microbiol 2018;41:95–105.Search in Google Scholar
24. Ritrovato, M, Faggiano, F, Andellini, M, Bassanelli, E, Esposito, MT, Cristiano, I, et al.. Aumentare la consapevolezza delle decisioni in ambito sanitario: il metodo Decision-oriented HTA. GIHTAD 2021;14:3.Search in Google Scholar
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/cclm-2022-0262).
© 2022 Walter de Gruyter GmbH, Berlin/Boston
