Utility of preoperative polymerase chain reaction testing during SARS-CoV-2 pandemic: The challenge of evolving incidence

Abstract

Introduction

Due to its high transmissibility, measures aimed at reducing the spread of SARS CoV2 have become mandatory. Different organizations have recommended performing polymerase chain reaction tests (PCR) as part of the preoperative screening of surgical patients. We aimed to determine the performance of PCR testing to detect asymptomatic carriers.

Methods

Observational study carried out at a tertiary care center. We compared the results of preoperative real-time reverse-transcription-PCR test (RT-PCR) performed on a cohort of patients pending surgery with the results we would have expected assuming the epidemiological data released by government offices.

Results

We registered no positives in the 2,722 preoperative RT-PCR tests performed in our health care area between epidemiological Weeks 18 to 21, meaning a cumulative incidence trending to zero. Assuming public epidemiological data, the probabilistic projection of potential asymptomatic individuals ranged from 0.27 × 10e⁻⁴ (according to official data of new cases diagnosed by PCR) to 4.69 × 10e⁻⁴ if we assumed cases confirmed by IgG test in our province. Assuming a RT-PCR sensitivity of 95%, to obtain a positive result we should perform 38,461 and 2,028 tests respectively.

Conclusions

In scenarios of very low prevalence and despite high sensitivity scores, indiscriminate preoperative RT-PCR screening is of a questionable effectiveness for detecting asymptomatic carriers. Our findings evidence the difficulty of establishing reliable predictive models for the episodic and rapidly evolving incidence of infections such as has characterized the SARS CoV2 pandemic.

Resumen

Antecedentes y objetivo

La alta transmisibilidad de la infección por SARS CoV2 ha obligado a los sistemas de salud mundiales a arbitrar medidas para evitar su expansión. En España, el consenso alcanzado entre diferentes sociedades científicas recomienda la realización de la prueba de reacción en cadena de la polimerasa (PCR) como cribado preoperatorio de portadores asintomáticos. Nos propusimos evaluar el rendimiento de la PCR preoperatoria para detectar portadores asintomáticos.

Material y métodos

Estudio observacional realizado en un hospital de tercer nivel. Comparamos los resultados de la prueba de PCR en tiempo real (RT-PCR) realizada en una cohorte de pacientes quirúrgicos de nuestra área asistencial con los resultados que hubiéramos esperado asumiendo los datos epidemiológicos publicados por las oficinas gubernamentales.

Resultados

No registramos resultados positivos en las 2722 RT-PCR realizadas en nuestra área entre las semanas epidemiológicas 18 a 21, lo que implica una incidencia acumulada de nuevos casos tendente a cero. Asumiendo los datos epidemiológicos publicados, la proyección probabilística de individuos asintomáticos varió de 0.27 × 10e⁻⁴ (datos oficiales de nuevos casos diagnosticados por PCR) a 4.69 × 10e⁻⁴ si asumimos casos confirmados por IgG en nuestra provincia. Suponiendo una sensibilidad de RT-PCR del 95%, para obtener un resultado positivo, deberíamos realizar 38,461 y 2,028 pruebas respectivamente.

Conclusiones

En escenarios de muy baja prevalencia y a pesar de su alta sensibilidad, la detección preoperatoria de portadores asintomáticos mediante de RT-PCR es de una efectividad cuestionable. Nuestros hallazgos evidencian la dificultad de establecer modelos predictivos fiables en el contexto de epidemias de evolución rápida, como la pandemia de SARS CoV2.

Introduction

The high transmissibility of SARS-CoV-2 and its significant impact on the elderly have presented healthcare systems worldwide with a major challenge that has led to measures aimed at reducing the spread of the disease.^1–3 In the case of surgical patients, different methods have been introduced to identify asymptomatic carriers, a basic step in the control of nosocomial transmission. Several scientific societies, consensus panels, and government agencies have recommended including polymerase chain reaction (PCR) testing in the preoperative screening of patients undergoing elective or urgent surgery.^2,4–6

Following the recommendations published in the context of the pandemic,^2,4–6 all patients awaiting surgery are systematically screened for infection in our hospital. Between 48 and 72 h before surgery, patients are contacted by phone by a nurse to assess their potential risk of being infected and the presence of COVID-compatible symptoms. If the telephone interview is negative (Fig. 1), patients undergo real-time PCR (RT-PCR) 24–48 h before surgery. If the telephone interview is positive, surgery is postponed, the patient is referred to the public health services and the corresponding diagnostic and quarantine measures are activated (Fig. 1). Surgery is also postponed and primary care services notified if the preoperative RT-PCR test is positive.

Figure 1.

Preoperative SARS CoV2 screening algorithm for elective surgery.

We define asymptomatic carriers as individuals who do not present symptoms in the preliminary telephone interview, but present a positive result in the RT-PCR test.

The aim of this study has been to determine the usefulness of performing RT-PCR tests before surgery. We compared the diagnostic yield of all preoperative RT-PCR tests performed in our hospital area up to the time of writing with the results that, in theory, should have been obtained based on the Spanish government’s latest epidemiological data published.

Methods

This was a retrospective observational study conducted in a tertiary university hospital formed of 5 independent centres that include a paediatric hospital and a major outpatient surgery unit. We collected the results of all preoperative RT-PCR tests performed in patients scheduled for surgery in our catchment area up to the time of writing.^6,7 All data were anonymous and were treated as discrete variables (number of positive RT-PCR tests out of the total number of RT-PCR tests performed). The study was approved by the hospital’s research ethics committee on 12 June 2020 (secretary Dr. Carlos García Pérez).

Preoperative RT-PCR

SARS-CoV-2 RNA testing was carried out on samples obtained from nasopharyngeal swabs. All swab samples were obtained by specially trained nurses in facilities designed for this purpose located in our catchment area. All samples were analysed in our central laboratory.

The RT-PCR used in our laboratory was the cobas® SARS-CoV-2 Test (Roche Molecular Diagnostics, Pleasanton, CA, USA), a fully automated sample-to-result system for qualitative detection of ORF1 and part of the pan-sarbecovirus E gene. A positive SARS-CoV-2 test was defined as detection of the ORF1 gene or the ORF1 and E genes. If the test was only positive for the E gene, a second RT-PCR test was performed. Briefly, we automatically extracted nucleic acids on a MagNa Pure Compact system (Roche Applied Science, Mannheim, Germany) using the MagNA Pure Compact Nucleic Acid Isolation Kit I, in accordance with the manufacturer's instructions. To detect SARS-CoV-2, we used the LightMix® Modular SARS gene kit and Wuhan CoV E-gene, RdRP-gene and N-gene kits (TIB Molbiol, Berlin, Germany) with LightCycler Multiplex RNA Virus Master (Roche, Basel, Switzerland), in accordance with the manufacturer's instructions. A sample was positive for SARS-CoV-2 if at least 2 genes were detected. If the sample was positive for only 1 gene, the test was reported as inconclusive.

According to reports, asymptomatic or minimally symptomatic individuals excrete enough viral genetic material through the upper respiratory tract between 2 and 14 days after infection to be potentially detected by RT-PCR testing of a nasopharyngeal swab.^2,7,8 The size of the viral load required for the carrier to be considered infectious has yet to be clearly defined. Some authors have reported that carriers with viral loads of less than 10⁵ gene copies per nasopharyngeal sample have a very low infectious potential, with the viral load being dependent on the time elapsed since infection.^7,9 In the absence of more definitive data, we analysed out data based on the assumption that patients with detectable viral genetic material in a nasopharyngeal sample are infective regardless of the amount of that viral material. Assuming a sensitivity of 95% and a specificity of 100%,¹⁰ RT-PCR results from correctly collected samples would detect 95% of asymptomatic carriers and would not generate false positives.

Estimation of asymptomatic carriers on the basis of official epidemiological data

We used 2 probabilistic projections to estimate the number of asymptomatic carriers in our catchment area. Our objective was to determine which of the 2 predictive models more closely matched the real data obtained from our sample, which included the results of all preoperative RT-PCR tests performed in our laboratory between 1 and 21 May 2020 (epidemic weeks 18–21). Accumulated incidence rates were estimated for a population of 1,942,389 individuals, corresponding to the registered inhabitants of the province of Seville.¹¹

We collected official epidemiological data published by government agencies (Ministry of Health and Health Council of the Junta de Andalucía) on the number of new cases (individuals with a positive PCR test) in the province of Seville 12 days prior to performing the nasopharyngeal swab; that is, data from epidemic weeks 17–19 were collected. Using this number, we estimated the average cumulative incidence of new cases during the study period, and constructed contingency tables based on the sensitivity of the test and 2 hypothetical simulations: Simulation 1: we assumed that the official cumulative incidence data basically refers to symptomatic individuals, in which case we estimated a ratio of symptomatic versus asymptomatic individuals of 1: 0.31 (based on previously published studies)¹²; Simulation 2: we estimated the number of asymptomatic individuals by comparing the official figures for the total number of individuals with a positive PCR result until 21 May 2020 with the number of serologically positive individuals (presence of IgG + in blood tests) in our province according the prevalence study published by the Spanish Ministry of Health.¹³ Based on these data, in Simulation 2 we estimated a ratio of positive PCR individuals versus serologically positive individuals of 1:17 for the province of Seville (Fig. 2). We estimated a cumulative incidence of asymptomatic individuals of 0.27 × 10e⁻⁴ and 4.69 × 10e⁻⁴ for Simulations 1 and 2, respectively (Fig. 2).

Figure 2.

Estimated number of asymptomatic carriers and diagnostic yield of preoperative RT-PCR tests based on official epidemiological data for the study period in the province of Seville.

Results

None of the 2722 preoperative RT-PCR tests performed in our health area between epidemiological weeks 18–21 were positive.

According to official data, the cumulative incidence of new cases (confirmed by PCR) in the province of Seville until week 21 was 3056. Fig. 3 shows the rate of new cases per week.

Figure 3.

New cases (confirmed by PCR) per epidemiological week in the province of Seville. Data corresponding to epidemiological weeks 8 to 21, numbers correspond to weeks 17-19.

Fig. 2 shows the probabilistic projection of potentially asymptomatic individuals (presented as the cumulative incidence for the study period) and the number of tests that need to be performed to detect a positive case, assuming a sensitivity of 95%.

Discussion

If none of the RT-PCR tests performed in our hospital up to the time of writing were positive, we can assume that the rate of asymptomatic carriers among surgical patients in our catchment area over the study period was less than 1/2722, which gives a cumulative incidence of around 0, or, in the best-case scenario, less than 3.6 × 10e⁻⁴ for a sensitivity of 95%. This suggests that the number of asymptomatic carriers is lower than the figure based on the serological prevalence of infected individuals in our province¹³ (4.69 × 10e⁻⁴, Simulation 2).

It is important to note that the cumulative incidence estimates used in our model are based on recently published serological prevalence data for Spain, the results of which are preliminary.¹³ In the seroprevalence study carried out by the Spanish Ministry of Health, samples from 60,983 volunteers were analysed using the rapid IgG test and subsequently confirmed by SARS-COV-2 IgG antibody immunoassay in volunteers who agreed to a second round of testing (n = 16,953); both tests showed 97.3% agreement.¹³

In our study, we defined stable sensitivity and specificity values to compare the potential yield of the RT-PCR test in different epidemic prevalence scenarios. It is clear that the lower the sensitivity of a test, the poorer its diagnostic yield; therefore, for a constant cumulative incidence, the number of tests required to detect a positive increases as the sensitivity of that test decreases.

Establishing the cumulative incidence ranges needed to establish the cut-off point from which to reconsider our preoperative screening strategy is beyond the scope of this study; however, it is obvious that the lower the prevalence (cumulative incidence), the poorer the screening yield.

Our findings show the difficulty of establishing reliable predictive models for rapidly evolving epidemics such as that caused by SARS-CoV-2. Over the years, a number methods of estimating the number of infectious individuals in the context of emerging epidemics have been put forward,^14,15 and several authors have pointed out the difficulty of establishing accurate predictive models when some of the variables used in constructing these models have not been reliably estimated.¹⁵ This difficulty could be due to various factors. On the one hand, we have the absence of reliable data on the real number of infected individuals (symptomatic or not) at a given time, which is essential to define the pretest probability of developing a disease over a given period, and to construct reliable contingency tables. Characteristics such as the virulence of the virus, its clinical manifestations, the number and reliability of the diagnostic tests performed, or the quality of the epidemiological records used would also help explain these inconsistencies. On the other hand, we have factors such as the rapid evolution of the disease and the impact of the social or political measures adopted to control its spread and their corresponding effect on the epidemic curve. Either way, in epidemic events of this magnitude it seems logical to assume that the number of "real" cases exceeds the number of diagnosed cases. Some authors suggest that the number of undiagnosed cases could be as high as 18%–31% of the total,^8,12,16 especially in the early stages of an epidemic when testing is minimal.¹⁴ According to the study of serological prevalence in Spain, real cases could exceed official figures by 10–17 fold.¹³ In light of our results, we suspect that this "true" incidence accumulated on the upward slope of the epidemic curve.

In our study, we have assumed that the RT-PCR test has high specificity and sensitivity. Some authors downgrade the sensitivity of PCR tests, since their yield depends on the timing of the test. Grassly et al. suggested a sensitivity of 90% when the test is performed between days 3–21 after infection.¹⁷ According to other authors, the sensitivity of the test is affected by the quality of the sample.¹⁸ A lower specificity or sensitivity of RT-PCR testing will lead to a higher rate of false positives and false negatives. Given the low real incidence of SARS-CoV-2, even small a decrease in specificity could generate a considerable number of false positives, with their corresponding epidemiological, health and social consequences.

Another factor to be taken into account when evaluating the yield of our test is timing, in other words, the stage of the epidemic at which the screening policies were implemented. When screening is introduced on the downward slope of the epidemic curve, we must assume that yield will be progressively affected as the cumulative incidence decreases – simple but probabilistically true. The question is, therefore, at what stages of the upward and downward slope do we believe the performance of our diagnostic test to be justified?

One of the weaknesses of this study is the inclusion of only surgical patients or patients awaiting a diagnostic or therapeutic test under anaesthesia. Although we includes a wide variety of procedures, ranging from major outpatient surgery to endoscopy and paediatric surgery, our surgical cohort may not accurately represent the general population of our catchment area. Given their characteristics (elderly patients or patients with comorbidities, risks related to SARS-CoV-2, etc.), the patients in our sample would probably adhere more strictly to confinement measures, and would therefore present a lower prevalence of infection compared with the population as a whole.

Another weakness of this study is our treatment of cumulative incidence as a discrete variable measured weekly or biweekly. The growth rate of an epidemic event is actually a continuous variable that we tend to "discretize" on a daily or weekly basis. Therefore, the probability that a patient who undergoes a RT-PCR test on a given day is an asymptomatic carrier of SARS-CoV-2 is the result of the cumulative probability of having been infected between 2 and 14 days prior to testing. In this study, we simplified the model even more by taking the mean cumulative incidence in the 3 weeks prior to obtaining the nasopharyngeal sample, although we should point out that this cumulative incidence differed from the fortnightly incidence by less than 10% (Fig. 3). In short, using the accumulated incidence data from the preceding weeks while on the downward slope of the epidemiological curve probably led to an overestimation of the real incidence of new infections, while using the same data while on the upward slope would have underestimated the incidence. Either way, projecting the incidence of new cases on the curve would allow us to make the necessary adjustments, which should obviously be periodically checked against the real data.

Conclusions

We based our empirical estimates on inconsistent and rapidly evolving epidemiological data; however, our sample data probably give a more reliable picture of the real incidence of infections in our catchment area at the time of the study than other estimations. Our results show that despite the high sensitivity and specificity of the test, and a negative predictive value close to 100%, the usefulness of preoperative RT-PCR to detect asymptomatic carriers is questionable due to its poor diagnostic yield in very low prevalence scenarios (represented by cumulative incidence). The slope of the epidemic curve and local prevalence rates must be taken into consideration when planning the introduction of such screening policies.

Authors' note: at the time of writing (June 12, 2020, epidemic week 23), 4520 tests had been performed in our hospital. No positive results had been reported up to that date.

Conflict of interests

The authors have no conflict of interest to declare.

Acknowledgements

We would like to thank Dr. Trish Reynolds, MBBS, FRACP, from the Edanz Group (https://en-authorservices.edanzgroup.com/) for editing a draft of this manuscript.