Consensus method to update the GINF formulary request form

Abstract

Objective

To update the Guideline for the Introduction of New Drugs in the Formulary (GINF form) using the RAND/UCLA appropriateness method, which combines the best available evidence and an expert panel's judgement.

Study Design/Methods

Two procedures were employed to detect where improvements could be made to the former versions of the request form and to transform them into concrete scenarios, found from a telephone survey with GINF form users, and a structured review of the scientific literature. The list of scenarios was later assessed by an expert panel. In a series of successive rounds, the rest of the research team critically assessed the expert panel's result, applying a score.

Results

A total of 52 improvement proposals were registered; 31 of them dealt with the form structure and the remaining 21 referred to the form process. Six formulary request forms were selected from the literature review. The final version included 24 assessed scenarios mainly addressing clinical trials’ validity, qualitative assessment and local implications of the requested drug.

Conclusions

A new version of the GINF form has been developed. Much improvement has been made based on the guide users’ opinion, available evidence and similar experiences that have been carried out internationally. The whole process has been subject to the experts’ opinion following a contrasted, consensus methodology: RAND/UCLA appropriateness method.

Resumen

Objetivo

Diseñar una nueva versión de la Guía para la Introducción de Nuevos Fármacos (GINF), utilizando para ello la metodología RAND/UCLA sobre el uso adecuado, que combina la mejor evidencia disponible con el juicio de un panel de expertos.

Diseño del estudio/métodos

Se emplearon 2 procedimientos para detectar oportunidades de mejora de las versiones anteriores de la guía, que fueron transformadas en escenarios concretos: una encuesta telefónica a usuarios de la GINF, y una revisión estructurada de la literatura científica. Esta lista de escenarios fue evaluada por un panel de expertos mediante rondas sucesivas. El resto del equipo de investigación evaluó críticamente el resultado del panel de expertos.

Resultados

Se registraron 52 propuestas de mejora, 31 de ellas se refieren a la estructura de la guía y las 21 restantes se refieren al procedimiento de utilización de la guía. En cuanto a la búsqueda bibliográfica, 6 de las guías de inclusión de nuevos medicamentos fueron seleccionadas. La versión final incluyó 24 de los escenarios propuestos orientados principalmente a la validez del ensayo clínico, la evaluación cualitativa y las consecuencias locales del fármaco solicitado.

Conclusiones

La nueva versión de la guía GINF llevada a cabo incluye muchas mejoras extraídas tanto de la opinión de los usuarios de guía como de la mejor evidencia disponible y las experiencias similares que se han llevado a cabo a nivel internacional. Todo el proceso ha sido sometido a la opinión de los expertos tal como indica la metodología de consenso RAND/UCLA.

Introduction

Assessing and selecting drugs is one of the drug policy's main tools in hospitals and health centres. The increasing offer of new drugs, especially in certain therapeutic areas, and the lack of a comparative assessment in the registering process in most countries, means that hospitals have to make considerable effort when introducing new drugs into the healthcare practice.^1–3 This decision is made by the Pharmacy and Therapeutics Committee (P&TC), which uses a well-informed working methodology.^4–6

Nowadays, there are many factors causing the drug selection decision-making process to become increasingly more complex, such as the clear increase in formulary requests, pharmaceutical market pressure, and the difficulty of having objective, complete and updated information at the decisive moment.^7,8 The whole process added to high variability rates reported^9,10 in P&TC decisions, show that tools in drug selection must be standardised in hospitals.^11,12

The Andalusian Agency for Health Technology Assessment (AETSA) in collaboration with the Pharmacy Department from Virgen del Rocío Hospital developed an evidence-based application model in 2002. Its purpose was to make standardising tools and selecting drugs easier and to try and unify the criteria followed by different P&TC, naming it the Guideline for the Introduction of New Drugs in the Formulary (GINF).¹³

The GINF form is a questionnaire that is filled in by a hospital physician who wishes to make a formulary request. It is composed of four general sections: the most thoroughly developed of them is devoted to comparative evidences on efficacy, effectiveness and safety compared with alternatives available. The rest of the form includes general data about the drug, description of costs, and classification of requests according to the P&TC's decision. GINF has been incorporated as a quality standard by the Andalusian Health System, where it has been widely implemented.^14,15

Similar tools have been developed and implemented in other countries since the early 90 s. The PBAC (Pharmaceutical Benefits Advisory Committee) guide, applied in Australia,¹⁶ and the Academy of Managed Care Pharmacy guide, in United States, are notable for their methodological quality and their great impact.¹⁷ These guidelines, unlike GINF, are devised for pharmaceutical companies to request that a drug is included within a health system or in a hospital chain. This fact hinders its application to the conditions of a specific health centre.

Given the time elapsed since the first edition of this GINF form and after many informal requests to update it, AETSA has collaborated with the Pharmacy Department from Virgen del Rocío University Hospital to update it using contrasted methodology.

Our aim was to update the GINF form, using the RAND/UCLA appropriateness method, which combines the best available evidence and an expert panel's judgement.

Method

A research group, coordinated by AETSA, performed the GINF form updating process between September 2005 and December 2007. The RAND/UCLA methodology¹⁸ was employed, and the list of scenarios to be assessed by the expert panel was identified later on. The scenarios consisted of possible modifications to be introduced in the new guideline. We used two procedures to detect how the older versions of the form could be improved: a telephone survey with GINF form users, and a structured review of the scientific literature.

Telephone survey

The methodology and results of the telephone survey have already been published.¹⁹ We asked interviewees an open question and identified and systematically registered all the improvements proposed concerning the official version established by AETSA. We analysed the local changes that had already been made to the former guide drafts and produced a table of the proposals made. P&TC secretaries from all Andalusian public hospitals were interviewed during the first half of 2007. We also interviewed the people in charge of the Pharmacy Departments in the rest of Spain where there was evidence of GINF being used. The project researchers recorded the data, listing every modification according to the GINF section and sub-section, as well as to the question that it was referring to in the questionnaire.

Structured review of the literature

A structured review of the literature was conducted dealing with formulary request guidelines and procedures. To achieve this, we searched information on different sources, and extracted it in an orderly way. A Silverplatter interface search strategy was used on Medline to find relevant papers (Table 1). It maximised sensitivity given the difficulties encountered because of the topic and how extensive it is. There was no language restriction, and we included the period 1997-2007 in the search. In addition, we also conducted an Internet search on a range of assessment bodies’, scientific societies’ and health systems’ webpages for information related to the topic. We looked for new references in the documents found. Once the search had been completed, the documents were selected according to the following previously-established inclusion/exclusion criteria.

Table 1.

Silverplatter search strategy.

No.	Term
1	(Formular* and guidelin*) in Ti
2	(hospital? and formular*) in Ti
3	“Formularies”/ without-subheadings, standards
4	“Formularies-Hospital”/without-Subheadings,standards
5	“Drug-Approval”/ methods
6	#3 or #4 or #5
7	“Guideline-” in MIME,MJME,PT
8	Guideline-Adherence”/without-subheadings,standards, trends
9	“Evidence-Based-Medicine”/ all subheadings
10	“Decision-Making”/ without-subheadings
11	“Choice-Behavior”/ without-subheadings
12	“Decision-Making-Organizational” in MIME,MJME,PT
13	“Economics-Pharmaceutical”/ withoutsubheadings,standards, trends
14	“Cost-Benefit-Analysis”/ without-subheadings,methods, organization-and- administration, standards,trends
15	“Pharmacy-and-Therapeutics-Committee”/ withoutsubheadings,standards, trends, utilization
16	#10 or #11 or #12 or #13 or #14 or #15
17	#1 or #2
18	#8 or #9 or #10 or #11 or #12 or #13 or #14 or #15
19	(#6 in MJME) and #18
20	#17 or #19
21	#17 or #19

Inclusion criteria: papers referring to requesting documents or guidelines, and papers that dealt with requesting procedures. Exclusion criteria: papers addressing individual drugs or drug groups that did not provide relevant results with regard to requesting general methodology and papers dealing with registry request.

Two of the researchers applied the inclusion/exclusion criteria by reviewing the titles and abstracts of the papers, or the whole article in case of doubt. If there was disagreement on the references, they were reviewed later on jointly, and consensus was established. We identified the relevant aspects or issues that had not been included in GINF for every study and guideline. We also revised already-included aspects so as to find potential improvements.

Once we had completed the survey and the literature review, we compiled a list including all possible improvements that had been detected through both procedures, and a cause-effect figure was designed. The scenarios were grouped together into three sections: procedure modifications, overall guideline structure modifications, and specific section modifications. This last section was also subdivided into: petitioner's and drug's data; efficiency, effectiveness and safety; economic assessment; requests’ conclusions; and classification. Lastly, the researchers reviewed and drafted the scenarios in order to obtain relevant, feasible, and mutually exclusive changes.

The expert group consisted of professionals from different Spanish regions that had extensive experience in making decisions with regard to including drugs in the hospital formulary. It included physicians who had made a formulary request, P&TC members and assessors. The assessment was discussed during a group members’ meeting, which mainly focused on the scenarios upon which there was some disagreement during the first voting round. However, this caused some discussion for each of the scenarios, and all of the comments and/or suggestions made by some experts during the first round were debated. Afterwards, members were given the opportunity to change the original list of definitions, and delivered a new questionnaire including all the new changes proposed. Each one of the scenarios, during the meeting, was scored again individually and graded according to the scores.

In a series of successive rounds, the rest of the research team critically assessed the expert panel's result, applying a score. The resulting version was then subject to an external review performed by other Spanish health technology assessment agencies, which helped shape the final version.

Results

Scenario identification

We identified a target population of 31 Andalusian hospitals. Twenty-nine of them (93.50% of answers) completed the survey. In addition, we interviewed 10 hospitals from 9 other Spanish regions. Almost 80% of the interviewees suggested one or more changes for improvement. A total of 52 improvement proposals were registered; 31 concerned the guideline structure, and the remaining 21 referred to the form process. Each hospital provided between 0 and 6 proposals.

A total of 132 papers were retrieved from the bibliographic search. Fifteen original papers and 6 drug request guidelines were selected (Table 2). Every selected document was described in detail. A total of 27 improvement proposals were registered during this phase.

Table 2.

New drug request guidelines, found in the bibliographical search.

Country	Requesting guideline
Australia	Guidelines for the Pharmaceutical Industry on Preparation of Submissions to the Pharmaceutical Benefits Advisory Committee
Canada	Common Drug Review Submission Guidelines For Manufacturers
Scotland	Guidance to Manufacturers notes for Completion of the New Product Assessment Form
UK	NICE guidelines for manufacturers and sponsors
USA	Academy of Managed Care Pharmacy Format for Formulary Submissions
USA	Drug Submission Guidelines for New Products, New Indications and New Formulations

The cause-effect figure allowed us to observe how the various improvement opportunities affect how the guideline is used (Fig. 1). The research team reviewed the improvement proposals, finally producing a total of 46 scenarios to be assessed by the expert panel. The scenarios were organised into separate sections, according to the type of change and GINF section it referred to:

• –

  Procedure changes: requesting and assessing circuits, selective rejection of guidelines and recommendations to make its use easier. 11 scenarios.

• –

  Overall structural changes: products to facilitate filling-in, e-guideline, implementation handbooks, training resources on GINF methodology and translation into English. 10 scenarios.

• –

  Specific structural changes: new information on efficiency, effectiveness, safety and internal validity of clinical trials, new economic issues and more request-classification categories. 25 scenarios.

Figure 1.

Cause-effect diagram.

Expert panel development

We selected 12 experts, 9 men and 3 women, from 5 different Spanish regions. The group was composed of the following clinical specialities: 6 hospital pharmacists, 2 oncologists, 2 internal medicine specialists, 1 epidemiologist, and 1 clinical pharmacologist.

The expert's level of response to the first survey and their attendance to the second round was 75% and 83%, respectively. The level of appropriateness and agreement the experts granted to the whole set of scenarios, according to the answer rounds, can be seen in Table 3. After the first individual scoring round, 7 new scenarios were included, meaning that a total of 53 scenarios were assessed in the face-to-face meeting.

Table 3.

Expert panel scenario assessment (rows: level of agreement; columns: level of appropriateness).

	Appropriate	Uncertain	Inappropriate
1st round: 46 scenarios
Agreement	29	0	0
Disagreement	0	3	0
Undetermined	10	3	1
2nd round: 53 scenarios
Agreement	32	0	3
Disagreement	0	4	0
Undetermined	7	5	2

The final version was drafted by one of the researchers, after being appraised and put forward by the rest of the team through two successive consultation rounds. It included the 24 assessed scenarios considered appropriate from section 3 (specific structural modifications) after the second round, and all the uncertain scenarios from section 3. Exceptions were the conclusion summary, the inclusion/exclusion criteria, and impact on the Health Department markers. The main content included in the new guideline are summarised in the Table 4, according to the classification employed. This final version is available on AETSA's website,²⁰ and in the new interactive electronic guideline.

Table 4.

Main improvement lines included in the final version of the GINF guideline.

Chapter	Improvement lines
Procedure changes	Introducing a survey on the petitioners’ potential conflict of interests
	Recommending that the petitioner defend the GINF submission to the P&TC
Overall structure changes	Producing an integral diffusion strategy for successive guideline versions
	Producing an easily accessible electronic format with interactive aids
	Setting-up a webpage including all GINF methodology contents and relevant bibliography
Specific structural section changes	Introducing a thorough assessment of aspects related to the internal validity of the trials and the new drug safety
	Producing new sections and widening the already existing ones on the qualitative assessment of the applicant clinician with respect to the requested drug and the benefits the latter provides
	Including more questions related to local aspects of requesting the drug: incidence/prevalence of the disorder, the subgroups’ profile, diagnosing tests or additional tests in order to select/follow-up the subgroups
	Introducing a new question on possible impact at primary care level and changing those related to the number of target patients
	Introducing a new category to classify the requests. It allows the deadline to be assessed once more in 6 months or in the light of new evidence

Discussion

The main strength of this study was to develop a new drug approval request form and a methodology to update it; the RAND/UCLA-based methodology being the main contribution itself. Many improvement opportunities have been identified, which give grounds for updating GINF. The GINF form was updated, including changes that affect both the questionnaire format and its content. Most possible improvements are related to specific structural changes of the survey, mainly in the section concerning efficiency, effectiveness and safety.

This update also aimed to improve the form's quality without jeopardising the flexibility which makes the form applicable regardless of the type of hospital or the volume of drugs assessed. We therefore not only included new sections in the update process, but we removed unnecessary content and simplified headings.

The interviewees frequently showed concern that the guideline, which had initially been addressed to each hospital physician, was being systematically completed by the manufacturer of the requested drug. This will be brought to attention in successive GINF form versions. Moreover a recent study fostered by our team proved that there is a common (identical) draft for a set of five drugs in more than two thirds of the Andalusian hospitals.²¹ It is clear there is a need for channelling the communication between the pharmaceutical industry and the P&TC through a proper instrument.²² Although GINF has not been developed to meet that purpose, it has been devised as an educational tool to give rise to the necessary dialogue among the requesting clinicians and assessors. Therefore, devices which make the process easier should be developed.

The use of RAND-UCLA methodology is considered appropriate. It would be impossible to describe the whole methodology in this paper, which is why we have tried to reference additional papers to make the methods section more understandable. The technique is generally applied to surgical or medical procedures, but in this case, most of the criteria that were recommended in the topic selection are met (frequently used procedures, procedures using a considerable number of resources, high variability, and its controversial use), meaning that the impact caused by applying the appropriateness criteria is potentially high. The method employed, a modified Delphi technique, provides the expert group members with the opportunity to comment upon their appraisals in the various assessment rounds. The experience and contemporary bibliography on group processes point out that the possible bias introduced in a face-to-face group can be controlled, to a large extent, by efficient leadership.

One of the problems arising from the RAND/UCLA methodology is the potential variability of the results depending on the experts taking part. Moreover, it has been proven that this variability is higher when lower quality evidence is used. Due to the nature of the topic to be studied, we have mainly used observational studies and reviews. However, the similarity in the guides’ contents employed as reference, and the soundness of the results from the panel in subsequent rounds very much increase the process's external validity. The small sample size is among the limitations due to the application scope of the study. This scarce representativeness was partly corrected by including other hospitals from other Spanish regions.

Every hospital centre has been selecting drugs independently until recently, so there was little collaboration and coordination among centres. Nowadays, all the aspects that are linked to health information and decision-making are increasingly more interrelated.²³ Moreover, our context is very dynamic, and changes with regard to the role played by regulating agencies and administrator agents will undoubtedly lead to important changes in the next few years.¹²

Drug policy selection is an accepted marker of a centre's healthcare quality level for any type of health institution. The most important accreditation systems in the world require the existence of active drug selection policies in order to grant accreditation. In order to reach this quality level the whole process must be standardised. There are many tools to do so, such as normalised formulary request documents, standardised assessment reports, normalised P&TC working processes⁶, and therapeutic exchange programmes, among others.

This guideline's utility would mainly be reduced to a scope where new drugs were requested at a local level. This local nature would be a notable contribution of our tool, since it allows combining ‘evidence-based medicine’ with local knowledge and needs so that a reasonable decision can be made.²⁴ Nowadays, centralised processes used to assess new drugs are carried out in many countries. This is a step forward towards standardising processes; and it is a mandatory and common direction for different systems. However, this methodology has also been criticised and the debate concerning which the most appropriate level for decision-making is fails to cease.^25,26

The new tool's results, concerning utility and satisfaction must be compared to the current guideline. Documents used in other countries have undergone similar updating processes but as no results have been published, we are not able to establish a detailed comparison. We could have conducted either a literature review or a survey alone; the former might have rendered an extremely detailed tool but compromising the appropriateness and utility of the new version, while the latter could have achieved a largely supportive and user-friendly format but lack the contribution of new evidence.

A new version of the GINF form has been developed. Much improvement has been made, based on the guide users’ opinion, available evidence and similar experiences that have been carried out internationally. The whole process has been subject to the experts’ opinion following a contrasted, consensus methodology: RAND/UCLA appropriateness method.

Funding

This study has been financed by AETSA.

Conflict of interest

The authors have no conflict of interest to declare.

Expert panel:

Mariano Aguayo Canela. Head of Internal Medicine Department in Virgen Macarena University Hospital in Seville.

Joan Bautista Altimiras Ruiz. Head of Pharmacy Department and President of the Pharmacotherapeutic Commission. Parc Taulí Hospital, Sabadell.

M. Dolores Bejarano Rojas. Head of Pharmaceutical Supplies Department. Central Services of the Andalusian Health Service, Seville.

José Cabeza Barrera. Head of Pharmacy Department. San Cecilio Clinical Hospital, Granada.

José Ramón del Prado Llergo. Head of Pharmacy Department. Reina Sofía University Hospital, Cordoba.

Salvador Peiró Moreno. Public Health Research Center (CSISP), Valencia.

Francesc Puigventos Latorre. Chair of Genesis Group. Secretary of the P&TC. Son Espases University Hospital, Palma de Mallorca.

Teresa Requena Caturla. Hospital Pharmacy Department of La Paz University Hospital, Madrid.

José Manuel Varela Aguilar. Internal Medicine Service of Virgen del Rocío University Hospital, Seville. Group for Rational Use of Medicines.

External reviewers

José María Amate, Andalusian Agency for Health Technology Assessment. Instituto de Salud Carlos III. Ministry for Health and Consumer Affairs.

José Asua Batarrina. Head of the Health Technology Assessment Service, OSTEBA (Osasun Teknologien Ebaluazioko Zerbizua).

Iñaki Gutiérrez. OSTEBA (Osasun Teknologien Ebaluazioko Zerbizva).

Juan Antonio Blasco Amaro. Coordinator of Health Technology Assessment Department (UETS). Agencia Laín Entralgo.

Manuel Portela Romero. GP of Primary Health Care of the Galician Public Health Service. Sub-director of Pharmacy and Pharmacists.

Acknowledgements

We would like to thank Sara Gallego Villanueva and Antonio Romero Tabares for their collaboration in information tasks and data processing.