Point-of-care testing in primary healthcare: a scoring system to determine the frequency of performing internal quality control

Introduction

Physicians in primary healthcare base many of their clinical decisions on point-of-care testing (POCT) results. The ISO 22870:2016 defines POCT as “testing that is performed near or at the site of a patient with the result leading to possible change in the care of the patient” [1]. POCT aims to provide test results more quickly, which induces more expeditious clinical decisions than if analyses are performed in large medical laboratories. Despite ongoing improvements in technology and operational simplicity, POCT has many potential limitations [2]. Therefore, to ensure accurate and reliable test results, POCT guidelines describe comprehensive plans for equipment selection and maintenance, staff training, reporting results, external quality assurance (EQA), and internal quality control (IQC) [1]. However, publications are few and evidence scarce for determining when IQC should be used in POCT [3, 4].

IQC is analysis of control samples containing analytes of a known concentration to monitor the accuracy and precision of the analytical process over time. In contrast to IQC performed in large medical laboratories, analysis in primary healthcare is commonly handled by non-laboratory personnel on single-analyte devices and with a low number of samples. Another difference is that opposite analysis in large medical laboratories where most errors occur in the preanalytical phase, most errors in POCT occur in the analytical phase [5]. Therefore, the IQC procedures from large medical laboratories cannot be adopted, but must be specially designed to the POCT environment.

The ISO 15189:2012 states that quality control materials should be periodically examined with a frequency that is based on the stability of the procedure and the risk of harm to the patient from an erroneous result [6], while the ISO 22870:2016 states that the IQC frequency should be specified for each device [1]. In a study from our group, it was found that analysing internal quality control once a week for hemoglobin, C-reactive protein (CRP), and glucose was associated with better analytical quality [7]. Various POCT guidelines suggest that IQC should be performed before a new batch of reagents or tests are being used, after unexpected test results, and after maintenance and instrument repair [8], [9], [10], [11], [12], [13]. These guidelines also propose that IQC should be analyzed at least monthly [9, 10], weekly [8], daily [9, 12, 13], or at the frequency proposed by the manufacturer [9, 11, 12, 14]. However, these recommendations are neither universal nor device specific, meaning that each practice should determine the appropriate IQC frequency for the POCT devices available to them. This process can be challenging [15, 16]. Various studies have proposed different procedures for determining the IQC frequency and have applied a pragmatic approach that considers the risk of harm to the patient based on the importance of the analyte in diagnosing and monitoring patients, the complexity and user-friendliness of the POCT devices, and the number of patient samples [3, 16], [17], [18], [19]. To our knowledge, however, previous studies have not combined all these factors into a single scoring system for determining the IQC frequency.

The aim of this study was to develop a scoring system for determining the IQC frequency for POCT devices in primary healthcare.

Materials and methods

The scoring system

The general principle of the scoring system was that higher probabilities of erroneous test results and higher risk of harm to the patients mean that IQC should be performed more frequently. Four factors were considered to significantly impact IQC frequencies (Figure 1). The factors were as follows:

1. Risk of harm to the patient based on the importance of the analyte in diagnosing and monitoring patients.

2. Type of POCT device.

3. User-friendliness.

4. Number of patient samples analyzed over a specific period.

Figure 1:

The proposed consensus-based scoring system for determining internal quality control (IQC) frequency on point-of-care (POCT) devices in primary healthcare.

For each POCT device, factors A, B, and C were given a certain number of points (a score), and the total score was calculated as the sum of the three scores. The total scores determined the general recommended IQC frequency for each POCT device. In addition, the number of patient samples analyzed over a specific period determined whether and how to adjust these frequencies in each individual practice.

The distribution of points and the interpretation of total scores were made by the working group members based on theoretical knowledge and practical feasibility. The principles behind the assessments are described in the following.

Results

The general scoring system was applied to 17 analytes and 134 different POCT devices (153 analyte-device combinations in total), which were all devices used by more than five participants in the Noklus EQA schemes (Supplementary Table 2). Table 1 lists the scores for the most frequently used POCT devices.

Most of the devices analyzing high-risk analytes (71/74) ended up with an IQC frequency from the general scoring system of daily (n=14) or weekly (n=57). For INR analyzed on CUBE (Eurolyser Diagnostica GmbH, Salzburg, Austria) and Simple Simon PT (Zafena AB, Borensberg, Sweden), the suggested frequencies were daily, while for CoaguChek (Roche Diagnostics, Mannheim, Germany), iLine microINR (iLine Microsystems S.L., Gipuzkoa, Spain), and Xprecia Stride (Siemens Healthcare, Erlangen, Germany) suggested frequencies were weekly due to instrument complexity and user-friendliness scoring differently (Supplementary Table 2). All blood-cell counters and glucose meters ended up with IQC daily and weekly, respectively.

For all mononucleosis tests (n=6) and helicobacter pylori tests (n=3), and almost all pregnancy tests (16/18), the recommended IQC frequency was occasionally (Supplementary Table 2). However, if the number of patient samples is very low, such as 0–3 each month, IQC should be performed before every patient sample (Table 2).

The Afinion (Abbott, CA, USA) multiassay analyzer obtained different IQC frequencies for the available analytes: CRP, HbA_1c, and urine albumin-creatinine ratio (ACR) obtained weekly, whilst cholesterol obtained monthly. In contrast, all analytes measured in the multiassay analyzers Cobas b101 (Roche Diagnostics International Ltd, Rotkreuz, Switzerland), Cobas h232 (Roche Diagnostics International Ltd, Rotkreuz, Switzerland), DCA (Siemens Healthcare, Erlangen, Germany), CUBE, and QuikRead (Aidian Oy, Espoo, Finland) obtained the same frequency for all assays (Table 1, Supplementary Table 2).

Table 3 shows that more laboratories will be recommended to increase rather than decrease their present IQC frequency when using the scoring system. For example, data registered in conjunction with an EQA survey for HbA_1c provided by Noklus in November 2019 showed that 55% (n=413) of the laboratories using Afinion analyzed IQC with a frequency in accordance with the scoring system, 41% (n=308) would be recommended to increase the IQC frequency, and 5% (n=36) would be recommended to decrease the IQC frequency after implementation of the scoring system.

Discussion

This study presents a consensus-based proposal for a scoring system that aims to provide differentiated and device-specific IQC frequency recommendations based on a model that considers different risks associated with the reporting of incorrect patient test results. This scoring system provides specific recommendations for primary healthcare laboratories. Its development was based on a literature review by and the professional judgment of experienced Noklus working group members, along with refinement based on input from laboratory advisors and medical specialists in laboratory medicine within Noklus.

The risk of harm to the patient is a key factor in the ISO 15189:2012 [6]. Dividing analytes into two groups (moderate-risk and high-risk analytes) was challenging and may be controversial, since most tests with erroneous results can have serious consequences. We still believe that erroneous results from some analytes cause more harm than others, and we therefore categorized them based on what, to our knowledge, was the most common role of each analyte in primary healthcare consultations. The analyte scores and therefore frequency outcomes can easily be adjusted if clinicians assess the analytes differently within the scoring system. Analytes were categorized into two groups to indicate which analytes should be controlled more frequently than others, which is consistent with a collective opinion paper by Cooper et al. [19] arguing that tests with a high potential for harm to the patient should have more frequent IQC.

The categorization of POCT devices into four types in this study was based on the diversity of devices used in primary healthcare, and is similar to two previous studies categorizing devices into four types [3] and three types [17]. Our model considers type of device to be one of four factors that determine IQC frequency, while these previous studies primarily based IQC frequencies on only the device type [3, 17]. The technology of POCT instruments is constantly improving, with some having built-in electronic checks or optical test cartridges that detect and minimize the risk of analytical errors. The two previously mentioned studies have argued, similarly to others, that devices should have less frequent IQC intervals when they have built-in electronic checks or optical test cartridges run every day that the device is used [3, 17, 18, 29]. We also believe that these electronic checks may reduce analytical errors, but we have not included these in our scoring system as they, in our opinion, do not reduce the need for performing control samples due to them not controlling the entire analytical process [9, 18, 24, 29]. Our experience is also that package inserts and user manuals from the manufacturer often lack information on which internal components are checked and which acceptance thresholds are used. It was therefore impossible to determine which built-in checks have an impact on the test result quality. However, instruments with integral IQC systems such as the iLine microINR that perform IQC as part of analyzing each patient sample may not require further IQC.

Our scoring system suggested a weekly IQC frequency for glucose meters. This is less frequent than that proposed in two previous studies, which recommended the analysis of IQC on glucose meters every day on which the instrument is used due to a lack of built-in checks [3, 17]. However, this does agree with the results of a study conducted by Noklus, which indicated that one of the factors associated with good glucose meter performance was performing IQC weekly [7]. Previous studies have suggested that the number of tests performed must be considered when deciding an IQC frequency [3, 8, 19, 30]. We propose to increase the frequency to daily if the practice analyzes more than 50 glucose samples each week. For practices that collect 0–3 patient samples each month, the recommendation was to analyze IQC before each patient sample to ensure the correct performance of the POC system. These numbers can though be defined differently in other environments.

Operators that do not adhere to test procedures are a major source of error in POCT [21], and more difficult or comprehensive procedures have higher probabilities of creating errors and producing erroneous test results [18]. Therefore, the possibility of user error has been included as a separate factor in our general scoring system. Classifying ease of use into three groups is also done when the Scandinavian evaluation of laboratory equipment for primary healthcare (SKUP), which Noklus is a part of, evaluates user-friendliness in real-life environments in their POCT device assessments [20]. The working group has identified many potential errors in the preanalytical, analytical and postanalytical phases of POCT testing that generally apply to many primary healthcare laboratories. However, the general scoring system does not implement local conditions in the risk assessment, such as for example in an individualized quality control plan (IQCP) [23, 31]. However, in our opinion, making local adjustments to our general recommended IQC frequencies is something that is easy and thus feasible for the small primary healthcare laboratories to do rather than using more comprehensive tools themselves.

It could be argued that the device type and user-friendliness may be related, meaning that the least and most complex devices may be the easiest and most difficult to use, respectively. However, our study indicated that this is not the case. POCT devices that scored as moderately difficult to use ranged from the least complex (e.g., the qualitative group A streptococcus tests) to the most complex (e.g., blood-cell counters) device types.

Competence and training of personnel in analyzing patient samples and IQCs are important factors in reducing the risk of errors [1, 13, 18]. Therefore, practices with many operators may consider a local increase in IQC frequency to ensure staff user competence. It is also important that practices purchase POCT devices that are adapted to the knowledge and education of the employees in the practice [25]. Noklus evaluates device performance based on user manuals, SKUP evaluations, EQA-data, literature review and instrument-related problems reported by users and laboratory advisors.

Many large medical laboratories base their IQC frequencies and control rules on statistical quality control, such as six sigma and Westgard rules [32]. In theory, the six sigma calculations and Westgard rules can, of course, also be used in primary healthcare for POCT. In an ideal world, this would be preferable. However, our experience is that non-laboratory personnel find this cumbersome and difficult, and they are not easily motivated to for example calculate the analytical coefficient of variation (CV) and implement control rules. Therefore, in primary healthcare, we think that we must start pragmatically with simpler rules like for example accepting IQC if the control results are within the manufacturer’s limits found in the package inserts. Hence, the IQC procedures from large medical laboratories cannot be adopted but must be specially designed to the POCT environment. For the moment, we think it is important to educate users that IQC procedures are important to implement, and as a start this is what we think is possible and meaningful with the resources available in primary healthcare. In the future, after evaluating the current recommendations, we will examine more closely whether more advanced control rules should be implemented for POCT in primary healthcare.

A limitation of our study is that the scoring system is based on a consensus approach from only one organization, but we think that the system can be adapted to other environments if desired. However, it is important to emphasize that any national accreditation- or regulatory requirements on IQC frequency should be followed, and the frequency should not be less than what the manufacturer requires. The scoring system has not been validated to test if the frequency recommendations are adequate to prevent errors and reduce patient risk. It is, however, difficult to design such a study in any laboratory, not only for primary healthcare but also for IQC procedures in large medical laboratories. The present paper is an attempt to systemise some of our experience on this topic and at the same time build on the few previous publications in this field. Noklus is now implementing this recommendation in primary healthcare laboratories and will monitor the degree of implementation through quality indicators collected in conjunction with each Noklus EQA survey. Data obtained before introducing the scoring system showed that more laboratories should increase rather than decrease their IQC frequency when introducing this scoring system (Table 3).

IQC plays an important role in quality assurance. However, there is currently no universal consensus on obtaining device-specific recommendations for when IQC should be analyzed. Our proposed points-based scoring system is easy to use and may induce systematic and effective assessments of the important factors for deciding IQC frequency in primary healthcare.