Scientific evidence and research in antimicrobial stewardship

Abstract

Evaluating the impact of antibiotic stewardship programs is challenging. There is evidence that they are effective in terms of reducing the consumption and cost of antibiotics, although establishing their impact on antimicrobial resistance (beyond restrictive policies in outbreaks caused by specific antimicrobial resistant organisms) and clinical outcomes is more difficult. Proper definitions of exposure and outcome variables, the use of advanced and appropriate statistical analyses and well-designed quasi-experimental studies would more accurately support the conclusions. Cluster randomized trials should be used whenever possible and appropriate, although the limitations of this approach should also be acknowledged. These issues are reviewed in this paper. We conclude that there are good research opportunities in the field of antibiotic stewardship.

Resumen

La evaluación del impacto de los programas de optimización de uso de antibióticos supone un reto. A pesar de que hay evidencia de la eficacia de estos programas en la reducción del consumo y coste de los tratamientos antibióticos, establecer su impacto en la reducción de resistencias (más allá de determinadas intervenciones restrictivas en brotes causados por microorganismos resistentes concretos) y en mejorar los resultados clínicos es más difícil. Para poder establecer conclusiones sólidas se necesita, en general, una adecuada definición de las variables de exposición y resultado, el uso de técnicas de análisis avanzadas adecuadas al diseño y estudios cuasi-experimentales bien diseñados. Siempre que sea adecuado y factible debe intentarse realizar estudios aleatorizados de clusters, pero las limitaciones específicas de estos deben tenerse en cuenta. En este artículo se revisan estos aspectos, y se concluye que hay buenas oportunidades para la investigación en el área de los programas de optimización del uso de antibióticos.

The importance of research in antibiotic stewardship: opportunities and challenges

In recent years, the optimization of antimicrobial prescribing in hospitals in conjunction with parallel initiatives in the community setting has become a very important part of clinical activity in infectious diseases. Thus, in 2012, the Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC), the Spanish Society of Hospital Pharmacy and the Spanish Society of Preventive Medicine, Public Health and Hygiene joined forces to release a consensus document on PRograms for Optimizing the use of Antibiotics (PROA) in Spanish hospitals.¹ This was just the first step on the challenging path towards the goal of improving the quality of antimicrobial use. The efficacy and safety of antimicrobials should be measured as quality indicators, using both individual and population analyses to reflect their impact on the outcome of infection, the evolution of antimicrobial resistance and their economic impact on the health system.

The various interventions in any antimicrobial stewardship program may be difficult to implement, because they have to change the long-established attitudes of a large number of professionals. In prevalence studies carried out in Spain, as many as 42% of hospitalized patients were receiving antimicrobials in 2010 and 2011,² reflecting the large number of physicians prescribing antimicrobials. At the same time, individual attitudes are difficult to change, because they are based on personal beliefs and behavior, and sometimes on antiquated knowledge. The recommendations of programs directed at optimizing antimicrobial use must be supported by the best available evidence, which comes from well-designed randomized clinical trials, cohort studies, studies of antimicrobial resistance mechanisms and their relationship with antimicrobial use. The recommendations must also be supported by the best possible organization of education and intervention activities and by defining quality indicators for antimicrobial use. These topics are analyzed in the manuscripts of this issue of Enfermedades Infecciosas y Microbiología Clínica. An important way to support effective programs is by generating new knowledge on all the topics reviewed in this issue, particularly defining the best type of intervention in terms of clinical results and cost-effectiveness.

Infectious Diseases physicians and other clinicians who are experts in infectious diseases and antimicrobial use, microbiologists and pharmacists have the opportunity to contribute to improving their daily work and to general knowledge in the field though research on all the topics mentioned above. A PubMed Advanced Search (accessed April 29^th 2013) that included the term “antimicrobial stewardship” OR “antimicrobial policy” found 1325 publications between 2001 and 2006, and 2350 from 2007 to 2012. Research in this field presents significant and specific challenges. Among these challenges are the need to develop multidisciplinary and multicenter studies that compare interventions, to carry out powered population studies, to include complementary approaches to answering the various research questions arising from real clinical practice, to explore the problem of antimicrobial resistance, to evaluate education programs, and to determine the sustainability of various interventional approaches.

Scientific evidence for the effectiveness of activities aimed at improving the use of antibiotics

The fundamental goals of any antimicrobial stewardship program (ASP) are to monitor and direct antimicrobial use in healthcare institutions, thus providing a standard evidence-based approach to judicious antimicrobial use.^2–4 These goals should form part of the institutional strategies of hospitals and are greatly appreciated by professionals, administrators and society at large.^5,6

The primary objectives of an ASP can be summarized as follows: 1) to improve the clinical outcomes of patients by reducing potentially adverse drug events (such as Clostridium difficile-associated disease [CDAD]), morbidity, mortality, length of hospitalization, and healthcare-related costs; and 2) to prevent and/or reduce antimicrobial resistance.^1,6,7 These objectives are achieved through improvements in the quality of antibiotic use and reductions in exposure to antimicrobials.

The relationship between inappropriate antimicrobial consumption and the development, persistence and spread of antibiotic resistance has been evaluated in numerous published scientific papers. It is clear, however, that the factors associated with antibiotic resistance are complex, often corresponding to multiple interrelated phenomena, which makes it difficult to attribute a significant change in antibiotic resistance exclusively to the particular ASP intervention. There are also significant methodological problems involved in analyzing the impact of ASP from a causal point of view.⁸ in many interventions, the appropriate use or restriction of certain antibiotics is associated with widespread practices of infection control, such as promoting hand hygiene among staff or preventing transmission, which provide an extra benefit of ASP interventions. On the other hand, if the ASP is not accompanied by adequate infection control standards, transmission of some resistant pathogens that are less influenced by antibiotic use may continue despite an improvement in the quality of prescriptions. However, numerous published experiences have shown that the application of an ASP can help to prevent or control the spread of some drug-resistant organisms, especially Gram-negative rods or glycopeptide-resistant enterococci^9–14 and C. difficile.^15–18

The impact of appropriate antimicrobial use on improving the clinical outcomes of patients may seem obvious, yet the causal relationship is also difficult to prove. There have been numerous studies on the effectiveness of ASPs on clinical and microbiological outcomes in patients with bacteremia or Gram-negative rod infections. The appropriate use of antimicrobials has also been associated with a marked reduction in drug-related adverse events, particularly Clostridium difficile-associated disease (CDAD).^9,15,19,20 The most relevant marker for this important objective of an ASP is a reduction in mortality as a direct result of improved patient care and outcomes. The majority of the studies, however, were not designed to evaluate this indicator. To obtain strong evidence for this association, randomized, controlled multicenter studies are needed. Because ASPs are usually aimed at reducing antimicrobial exposure, it is also equally important to demonstrate that they are not associated with deleterious effects, which have been shown in various studies.²¹

A recent systematic literature review analyzed 66 studies of interventions (randomized controlled clinical trials, controlled before–and-after studies and interrupted time series studies), designed to improve antibiotic prescribing practices for hospital inpatients. The objective of the review was to assess the effectiveness of the interventions and to evaluate their impact on reducing the incidence of antimicrobial resistant pathogens or CDAD and clinical outcome. Fifty-one (77%) of the studies showed significant improvements in at least one of the objectives of the predetermined outcomes. In 60 studies, the aim was to reduce prescribed antibiotic treatment; 47 of them assessed a “drug outcome”, and 38 detected a significant improvement (81%); 16 studies evaluated a “microbiological outcome”, of which 12 (75%) improved significantly; in 9 studies, a “clinical outcome” was evaluated, in which only 2 (22%) showed a significant deterioration and 3 (33%) showed a significant improvement. The authors concluded that intervention programs for improving antimicrobial prescribing in hospitals are successful and can help reduce or control antibiotic resistant organisms and hospital-acquired infections, although the impact on variables associated with clinical outcomes is more moderate and difficult to assess.²²

Interventions that can be measured, confounders, and endpoints

Stewardship programs, like all quality programs, must be monitored using quality indicators that can be broken down into structure, procedure and outcome indicators. These indicators can also be used as endpoints for research studies. Changes in antimicrobial consumption, which is regarded as a procedure indicator, is one of the most common quality indicators measured in studies of stewardship programs. Consumption should be measured using generally accepted units, the most widespread of which is the defined daily dose (DDD). However there are others, such as the prescribed daily dose, which can be used as an alternative or a complement.¹ They should be calculated with a denominator such as 100 or 1000 patient-days and can be useful for benchmarking, although the challenges of case-mix differences are far from being resolved.^1,23 Other procedure indicators include the rate of adequate empirical therapy, the rate of adequate duration of antimicrobial therapy and the rate of de-escalation or optimized therapy (Table 1).

Table 1.

Some frequently used quality indicators and outcome variables for assessing the impact of antimicrobial stewardship programs

Indicator	Type of indicator	Example	Comments
Expenditure	Outcome, economic	Money spent purchasing, dispensing or administering antibiotics per admissions/patientdays	Cost of purchased drugs is easy to obtain
Antibiotic consumption	Procedure, consumption	DDD or PDD per admission or patient-days	Potentially useful for benchmarking (need to adjust for case-mix)
Length of treatment	Procedure	Average duration of treatment (days) for specific infections	Need to collect individual data
Appropriate therapy	Procedure	Proportion of appropriate regimens according to guidelines	Need to collect individual data
Rate of de-escalation	Procedure	Proportion of de-escalated regimens	No consensus definitions
In-hospital mortality	Outcome, clinical	Crude mortality for specific infections	Attributable mortality is less difficult to define
Length of stay	Outcome, clinical	Average length of stay of patients	Need to adjust for case-mix
Rate of readmission	Outcome, clinical	Rate of readmission	Need to adjust for case-mix
Antibiotic- resistant organisms	Outcome, microbiological	Percentage resistance or rates of resistant pathogens	Key resistance profiles and pathogens must be considered
Clostridium difficile	Outcome, adverse events	Rate of C. difficile-associated disease	Influence of active search for cases must be considered

DDD: defined daily doses; PDD: prescribed daily doses.

Inappropriate antibiotic treatment is common, especially in hospital-acquired infections, and an increase in appropriate treatment during an interventional study is one of the parameters usually assessed in any ASP. Programs that reduce overall antimicrobial usage by minimizing inappropriate or lengthy use have the potential to reduce the risk of drug-related adverse events. There is a growing body of evidence demonstrating that antimicrobial stewardship programs modify the quantity and quality of antimicrobial prescriptions and reduce the expense burden.^1,22,24,25

However, outcome measures are of greater interest since they reflect all aspects of care and are the ultimate objectives of the intervention. The choice of outcome variables in antimicrobial stewardship programs varies widely depending on program design, duration and goals. Obviously, measurement capabilities can also be taken into account. A meticulous evaluation of outcome measures is clearly needed to ensure that these efforts are sustained.²⁶ Outcome indicators can be grouped into two categories: clinical outcomes and microbiological results (Table 1).

Clinical outcomes usually include crude mortality, duration of hospitalization, and readmission rates. Infection-related mortality and clinical cure and improvement are less frequently used because it is more difficult to define these terms precisely and they are more subject to interpretation. Unfortunately, it is also difficult to demonstrate the impact of these programs on mortality, because they frequently are not powered enough to reveal whether they are beneficial or detrimental to clinical outcomes that are infrequent (all-cause mortality), particularly when potential confounders are considered.²⁷ However, depending on the objective of the ASP, demonstrating that the mortality rate is maintained when an interventional program reduces antimicrobial use may be as important.

Several indicators have been measured to evaluate the impact on microbiological variables. Obviously, the choice of microbiological indicators depends on local ecology and problematic pathogens in the institution where the program is implemented. The prevalence of infection caused by difficult-to-treat pathogens such as methicillin-resistant Staphylococcus aureus, extended-spectrum β-lactamase–producing Escherichia coli and Klebsiella species, Pseudomonas aeruginosa or Acinetobacter baumannii, has been used extensively.²⁶ As explained above, other studies have evaluated the impact of antibiotic stewardship on CDAD rates, a common adverse event related to the prolonged administration of antibiotics.

The impact of antimicrobial stewardship programs on bacterial resistance can be difficult to assess due to the multiple factors that can influence resistance development and spread. Optimized antimicrobial use is thought to help reduce the emergence of resistance, although few prospective randomized trials have attempted to study this particular association.

Study designs: an overview

The association between antibiotic use and resistance can be analyzed with various study designs. Most studies use ecological designs, which aggregate (group-level) exposure and outcome data from various areas, hospitals or wards. The most basic example is an analysis of antibiotic consumption and bacterial resistance between different countries.^28,29 Such a design has significant limitations in terms of establishing causality: first, there is always the risk of committing the ecological bias or fallacy (meaning that the ecological association does not exist at the individual level); second, exposure and outcome are measured at the same time, with no information about change over time; and third, control for confounding is not usually achieved. A further step includes the use of time-series analyses, which is explained in more detail below. Finally, the association between antibiotic use and resistance can be investigated at the individual level using case-control or cohort studies, which may provide different results when compared with ecological studies.³⁰ The development of multilevel analyses is an attractive way to improve integration between individual and group levels.³¹

The impact of an intervention is also usually measured using aggregate data. Most analyses use a quasi-experimental design in which the intervention is not randomized. This concept includes before-after designs, interrupted time-series analyses, and studies with non-randomized concurrent control groups.³² The most advanced designs randomize the intervention. Because many interventions cannot be applied to individuals, only to groups (e.g., wards or hospitals), or because of the nature of the study or to avoid ‘contamination’ from non-intervention individuals, the group is the unit of randomization. This is the case of cluster-randomized trials. Eventually, the intervention can be applied to individuals and randomized clinical trials can then be used.

Quasi-experimental studies: before-after studies

In the field of infectious diseases, quasi-experimental study designs, also known as before-after studies or pre-post intervention studies, have been widely used for evaluating activities related to infection control and antibiotic resistance. In general, with this type of epidemiological study design it is possible to study the benefits and draw conclusions about the possible effect of an intervention on an infectious process, evaluate its impact on the emergence or spread of antimicrobial resistance and on the appropriate use of antibiotics. This design type is frequently used when it is logistically or even ethically impossible to conduct a randomized study or a controlled clinical trial.

There are numerous examples of before-after quasi-experimental studies, in which a specific problem situation (such as rates of antibiotic resistance in one or more organisms, CDAD or surgical site infection) is analyzed before and after the application of a specific intervention program (such as promoting hand hygiene, or imposing certain restrictions for therapeutic or prophylactic antimicrobial use). One of the essential aims of these studies, when applied to infectious diseases, is to demonstrate that a specific action is responsible for modifying the results or outcomes of a particular infectious process, microbial resistance level or the appropriate use of antimicrobials.

Although there are different types of quasi-experimental designs, two types are most frequently used in infectious diseases: those conducted without a control group (category 1) and those which, after prior assessment of the problem, use a control group during the intervention (category 2). To establish causality more definitively, it is necessary to frame a study in the second category.^32–34

The first category of quasi-experimental studies comprises 5 different designs, which do not include a control group, shown here in ascending order of complexity:

• –

  A single measurement of the outcome variable is performed before and after the intervention. This design, of great simplicity, rarely provides enough evidence of the effect of the intervention on the outcome.

• –

  Two measurements of the outcome variable are taken before the intervention, separated by an appropriate time interval, and one measurement is taken after the intervention. This type of design is intended to avoid the maturation effect, possible seasonal variations or regression to the mean.

• –

  Designs that include a non-equivalent variable, not directly linked to the intervention, along with the primary dependent variable. Both variables should assess similar constructs and thus be affected by similar confounding factors apart from the intervention; the primary dependent variable, therefore, is expected to change with the intervention, but not the non-equivalent variable.

• –

  A design in which four observations are made: the first two before the intervention, the third one after it, and the fourth once the intervention is removed. This type of design seeks to prove the hypothesis that the proposed intervention requires continuous implementation in order to maintain the desired effect.

• –

  The most complex design also incorporates four observations: the first before the intervention, the second after, the third after the intervention is removed, and the fourth after the intervention is implemented again. This type of study, which raises possible ethical issues, serves to demonstrate the reproducibility of the relationship between intervention and outcome. As explained in the third design, this type would be appropriate when the effect of the intervention may be transient and a strong causal link is suspected between it and the expected outcome.

The second category of quasi-experimental study types includes a control group. The control group is not randomized beforehand, but forms a comparison with the evaluation group. There are 3 subtypes in this category, which are, in ascending order of complexity:

• –

  Designs that select a control group in which no intervention is made. The same dependent variable is measured in both intervention and control groups. Avoiding selection bias is a major challenge for this kind of design.

• –

  Designs that are identical to the previous one, but that incorporate a double pre-test measurement in both groups before the intervention is implemented in the study group. This design enables an evaluation of the impact of confounding variables associated with the time period to be made.

• –

  Complex designs that evaluate the intervention after one measurement in the study group and after 2 measurements in the control group.

There are significant intrinsic limitations to quasi-experimental designs, some of which can be minimized depending on the particular design used. Such difficulties include differentiating the natural evolution of events and the effect of the intervention, insufficient measurements to demonstrate a trend and difficulties in controlling for confounders.

Time series analysis

A time series is a sequence of measurements taken at successive, usually regularly spaced, points in time. In this area, antibiotic consumption and resistance are typically measured at fixed intervals (monthly, bimonthly, quarterly, etc.). Time series analyses use mathematical models to predict the behavior of a variable (prediction), on the basis of previously observed values. For example, we may try to predict the evolution of a specific microorganism's resistance to a particular antibiotic in accordance with an observed relationship over time between the consumption of the antibiotic and resistance rates. There are various methods of performing time series analyses, although the most commonly used in the area of antibiotic use and resistance is the autoregressive integrated moving average (ARIMA), which uses a parametric approach. The relationship between several times series (in our example, antibiotic consumption and resistance rates) can be studied by transfer function models and by creating a multivariate time series analysis.^33,35,36

There must be sufficient observations in a time series analysis to accurately predict the behavior of the variable, and the periodicity of the measurements should bear in mind the potential influence of seasonality and the expected latency between an intervention and a change in the outcome measure. Some intrinsic limitations of time series analyses should be considered, including the use of aggregate data and taking a simplistic view of a very complex relationship, which can be improved by collecting time series of potential confounders. However, such models are very useful for exploring potential relationships and helping choose areas for intervention.

An alternative approach to a time series analysis for evaluating an intervention implemented at a specific moment (interrupted time-series analysis) is the so-called segmented regression analysis.^33,37 With this approach it is also necessary to obtain sufficient measurements of the outcome variable before and after the intervention. This method allows for the inclusion of potential confounders, if measured before and after the intervention, as well as an analysis of subsequent interventions.³⁸

Cluster randomized trials

As mentioned above, the most advanced designs for evaluating interventions are randomized, which theoretically avoids the effect of measured and unmeasured confounders. Usually, interventions to improve antibiotic use cannot be applied to individuals, but only to groups of individuals (clusters); sometimes, individual applications may not be suitable because individuals allocated to a control group may be ‘contaminated’ by the intervention (for example, if the impact of an education program is being assessed, it is likely that physicians allocated to non-intervention will comment on their cases with others receiving the intervention). In this type of case, the unit of allocation is not the individual but the cluster. The unit of analysis may be the cluster or the individual, and the unit of inference may also be the cluster or individual. The fact that individuals are not independent within the cluster should be considered.

Cluster randomized trials are more complex to design and analyze and require more participants than an individual cluster to be included in order to obtain similar statistical power; more power is gained, however, by including more clusters than by including clusters with more individuals. The size of the clusters must also be calculated,³⁹ and the main practical problem is finding sufficient numbers of similar clusters. For this reason, this design is increasingly being used in primary care centers, which may be more homogeneous, and critical care units. When different types of wards or healthcare centers must be included, potential solutions are matching clusters on the basis of specific features or stratifying their allocation by well-defined variables (although both approaches also have their problems). Allocations to the intervention may continue during the entire study period or may be subject to crossover, meaning that clusters initially allocated to non-intervention will start the intervention later, while the intervention ceases in those initially allocated to it.⁴⁰

In the hierarchy of designs, cluster randomized trials provide the highest quality of evidence in the field of ASP. However, some important problems of this design should be considered. Adherence to the intervention must be measured and be sufficiently high to avoid the non-differential bias caused by low adherence; if an impact on antimicrobial resistance or clinical outcomes is sought, the epidemiological situation and case-mix of each cluster must be considered.⁴¹

Recommendations for reporting cluster-randomized trials properly have been published.⁴²

Conclusions and future prospects

There are good research opportunities in the field of antibiotic stewardship. Well-designed and analyzed studies are needed in this area. Proper definitions of variables, the use of advanced and appropriate statistical analyses and well-designed quasi-experimental studies would support conclusions more accurately. Whenever possible and appropriate, cluster randomized trials should be used, although the limitations of this approach are also acknowledged.

Funding

Supported by the Ministerio de Economía y Competitividad, Instituto de Salud Carlos III - co-financed by European Development Regional Fund “A way to achieve Europe” ERDF, Spanish Network for Research on Infectious Diseases (REIPI RD12/0015).

Conflicts of interest

BA has received grant support from Gilead Sciences, Pfizer, and the Instituto de Salud Carlos III; and has been paid for talks on behalf of Gilead Sciences, Merck Sharp and Dohme, Pfizer, Astellas and Novartis.

AP has been a consultant for Merck and Pfizer, has served as a speaker for Wyeth, AstraZeneca, Merck, and Pfizer and has received research support from Merck, Pfizer and Wyeth.

JGM has served as a speaker for Astellas, Gilead Sciences, Merck Sharp and Dohme, and has received research support from the Instituto de Salud Carlos III and Wyeth.

JP has served as a speaker for Wyeth, Schering-Plough, MSD, Janssen, Novartis, and Pfizer and has received research grants from Novartis and AstraZeneca.

JRB has been a consultant for Wyeth, Merck, and Pfizer, has served as a speaker for Wyeth, Merck, Pfizer, Astra-Zeneca and GlaxoSmithKline, and has received research support from Merck and Wyeth.