Buscar en
Cirugía Española (English Edition)
Toda la web
Inicio Cirugía Española (English Edition) Surgery as a Scientific Discipline and the IDEAL Collaboration
Journal Information
Vol. 92. Issue 2.
Pages 71-73 (February 2014)
Vol. 92. Issue 2.
Pages 71-73 (February 2014)
DOI: 10.1016/j.cireng.2013.12.017
Full text access
Surgery as a Scientific Discipline and the IDEAL Collaboration
La cirugía como disciplina científica y la IDEAL Collaboration
Peter McCulloch
Nuffield Department of Surgical Science, John Radcliffe Hospital, Oxford University, Oxford, United Kingdom
Article information
Full Text
Download PDF
Tables (2)
Table 1. Defining Characteristics of IDEAL Framework Phases.
Table 2. Key Recommendations for Research Design at Each IDEAL Phase.
Show moreShow less
Full Text

Surgery has achieved tremendous things over the last century. Dramatic advances have included the development of cardiac surgery, transplant surgery and minimally invasive surgery, whilst improved peri-operative care has dramatically improved the outlook for patients.

Few of these advances, however, were based on randomised controlled trials (RCTs), and the number, size and quality of RCTs of surgical techniques remain much lower than for most medical disciplines. As this surgery has been severely criticised as unscientific, and surgeons have responded by claiming that there are special difficulties in performing such trials for surgical techniques. In 2009, the Ballioll Collaboration1–3 tried to analyse the problems of performing scientifically valid studies of surgery, and to propose solutions to them. This led to the development of the IDEAL Framework and Recommendations.

Most surgeons agree that RCTs are the most reliable method for comparing treatments, but most would also agree that there are difficulties in applying them to surgical operations. Many of our objections, however, when closely examined, turn out to be ill-founded. For example, surgeons often say that urgent life threatening illness or rare conditions prevent them from randomising patients −but RCTs for cardiac arrest resuscitation4 and for childhood leukaemia5 have both been done successfully. Of course surgeons cannot be blinded to the operation performed −but the surgeon should not evaluate his own results, and it is often possible to blind the person who does. Again, it is common for surgeons to claim that the benefit of their operations is so obvious that a trial would not only be unnecessary but unethical (the “RCT of parachutes” argument6). Unfortunately we surgeons are a self-selected group of very confident people, to whom things often appear “obvious” even when they later turn out to be wrong.7

However there are some genuine reasons why RCTs may sometimes be difficult, impossible, or simply not the right tool for evaluating surgery. First, surgical innovations are performed by an operator, whose skill is critical to success, and whose professional identity is closely tied to the results of her/his operations. Each patient, surgeon and operation is slightly different, and it is therefore more difficult to produce a standard definition of an operation than of a medicine. It is also difficult for a surgeon who has perfected one technique to go back to one he abandoned. Secondly, surgery always implies the risk of adverse outcomes, which are usually difficult or impossible to reverse. It therefore stimulates strong anxieties and preferences in both patients and surgeons, based mainly on the individual's attitude to risk. These two fundamental properties of surgery present difficulties for evaluation in different ways during the life cycle of a new operation. The IDEAL Framework describes the stages through which new surgical procedures develop (Table 1). The IDEAL Recommendations are suggestions for the types of studies, which should be conducted at each stage of the Framework (Table 2). The Framework does not reject RCTs, but it does recognise that there is often a lot of preliminary work to do before an RCT can occur.

Table 1.

Defining Characteristics of IDEAL Framework Phases.

Phase 1: Idea  Phase 2a: Development  Phase 2b: Exploration  Phase 3: Assessment  Phase 4: Long term monitoring 
Initial report  “Tinkering” (rapid iterative modification of technique and indications)  Technique now more stable  Gaining wide acceptance  Monitoring late and rare problems, changes in use 
Innovation may be planned, accidental or forced  Small experience from one centre  Replication by others  Considered as possible replacement for current treatment   
Focus on explanation and description  Focus on technical details and feasibility  Focus on adverse effects and potential benefits  Comparison against current best practice   
    Learning curves important     
    Definition and quality parameters developed     
Table 2.

Key Recommendations for Research Design at Each IDEAL Phase.

Idea: Professional innovation database  Development: Prospective development studies  Exploration: Phase IIS study  Assessment: Surgical RCT  Long term monitoring: Prospective registries 
Compulsory reporting of all new innovations  Detailed description of selection criteria  To evaluate technique prospectively and cooperatively  RCT – question agreed in Phase IIS  Should monitor indications as well as outcomes 
Confidential entry allowed to encourage reporting of failed innovations (similar to CHRP system)  Detailed technical description  To develop a consensus over definition of the procedure, quality standards and indications  Use power calculations from Phase IIS  SPC used for quality control (Shewart charts, CUSUM, VLAD) 
Hospital or institution to be informed separately as a professional duty  Prospective account of all cases consecutively, including those not treated with new technique/device  To gather data for power calculations  Use learning curve data to decide entry points for clinicians   
  Clear standardised definitions of outcomes reported  To evaluate and monitor learning curves  Use Phase IIS consensus to define operation, quality control AND outcome measures   
  Description of ALL modifications, and when they were made during the series  To achieve consensus on the trial questions  Use modified RCTs or recognised alternative if RCT not feasible:   
  Registration of PROTOCOL before study starts  To develop a multi-centre randomised trial (RCT)  Feasibility RCT   
      Expertise-based RCT   
  Use of Statistical Process Control (SPC) methods to evaluate progress    Cohort multiple RCT   
      Step-wedge design   
      Controlled-interrupted time series   

The first stage of the Framework (Idea) is the “First-in-man” study. Where the new procedure is planned, a protocol should be published in advance, but sometimes new techniques are developed in emergencies or even by accident. The key IDEAL Recommendation is simply that all potential “first-in-man” studies are registered in a database searchable by all health professionals. This will allow others to learn from success and prevent the repetition of failed or harmful procedures. To avoid the risk of litigation when innovations fail, confidential reporting should be allowed, following the good examples available in transport and energy production industries.8

In the second stage of the Framework, Development, the key feature is that the procedure is still being rapidly adapted and changed as the investigators learn from experience.9 Instead of the traditional case series, IDEAL recommends Prospective Development Studies at this stage. These are prospective cohort studies which report the outcome for every patient referred sequentially, whether or not they received the treatment. The reasons for rejecting patients and for changing the technique are reported as well as the point in the sequence when this occurred. This allows colleagues to understand how and why the technique has evolved, and avoids repeating errors.

By the third stage, Exploration, techniques are relatively stable, although there may be several alternative versions: they are being practised by several centres, and the number of cases performed is rising rapidly. This is the stage at which controversy has often prevented surgeons from performing an RCT. If surgeons are not agreed about the details of the procedure or the indications, and many are still on their learning curve, getting consensus on a patient group and a question for an RCT may be impossible. The IDEAL Recommendation is therefore a large prospective collaborative cohort study.10 Surgeons enter patient data using the indications and techniques they prefer, and this is all analysed and discussed after 1–2 years. This allows controversies to be resolved using data, and may also allow surgeons to demonstrate completion of their learning curve. This study type is called “Phase IIS” because of it's similarity to Phase II oncology studies. The aim is to help surgeons reach consensus on a question for an RCT, by reducing uncertainty about which treatment is better in which situation. The Phase IIS study is therefore often referred to as a “bridge” from small development studies to an RCT.

The last two stages of the IDEAL Framework are definitive comparison with current best treatment, preferably in an RCT, and long term surveillance, preferably using a registry.11 Getting surgeons and patients to agree to be randomised remain challenging, but there are several ways to make this easier. Patients can be followed up in parallel cohort groups if they refuse randomisation. Consent and randomisation can be done by trained nurses, instead of surgeons, who find it very hard to hide their personal biases. Or patients can be randomised not to particular operations but to particular surgeons, each of which then does their favoured operation (expertise based randomisation). Blinding of the observers who measure outcome is often possible.

Registries need to be carefully thought out by the professional community. They should collect a very minimal set of important data, but should do so very reliably. It is best (though often difficult) if all patients with a condition are included regardless of treatment: more commonly registries are based on a particular treatment, such as hip replacement. In such registries it is important to have some information about the numbers and types of patients who were not selected for treatment.

These suggestions for improving surgical research are now being adopted by various groups, who are trying out the new proposals for study design and reporting. This process will lead to further development, changes and improvements to the Recommendations. Regulatory authorities have also shown interest in using the IDEAL framework to devise better programmes for evaluating medical devices. The IDEAL Collaboration (http://www.ideal-collaboration.net/), an international group of surgeons and methodologists interested in promoting these ideas, are advocating changes to current practice, including the extinction of the case series, which should be replaced by Prospective Development Studies. We look forward to a future in which surgical research is conducted using tools appropriate to the job, including but not exclusively RCTs, and as a result surgery is more evidence based, safer, and more effective.

J.S. Barkun, J.K. Aronson, L.S. Feldman, G.J. Maddern, S.M. Strasberg, D.G. Altman, et al.
Evaluation and stages of surgical innovations.
Lancet, 374 (2009), pp. 1089-1096
P.L. Ergina, J.A. Cook, J.M. Blazeby, I. Boutron, P.A. Clavien, B.C. Reeves, et al.
Challenges in evaluating surgical innovation.
Lancet, 374 (2009), pp. 1097-1104
P. McCulloch, D.G. Altman, W.B. Campbell, D.R. Flum, P. Glasziou, J.C. Marshall, et al.
No surgical innovation without evaluation: the IDEAL recommendations.
Lancet, 374 (2009), pp. 1105-1112
C. Choux, P.Y. Gueugniaud, A. Barbieux, E. Pham, C. Lae, P.Y. Dubien, et al.
Standard doses versus repeated high doses of epinephrine in cardiac arrest outside the hospital.
Resuscitation, 29 (1995), pp. 3-9
D.J. Fernbach, K.M. Griffith, M.E. Haggard, T.M. Holcomb, W.W. Sutow, T.J. Vietti, et al.
Chemotherapy of acute leukemia in childhood. Comparison of cyclophosphamide and mercaptopurine.
N Engl J Med, 275 (1966), pp. 451-456
G.C. Smith, J.P. Pell.
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials.
BMJ, 327 (2003), pp. 1459-1461
P. McCulloch, A. Kaul, G.F. Wagstaff, J. Wheatcroft.
Tolerance of uncertainty, extroversion, neuroticism and attitudes to randomized controlled trials among surgeons and physicians.
Br J Surg, 92 (2005), pp. 1293-1297
M. O’Leary, S.L. Chappell.
Confidential incident reporting systems create vital awareness of safety problems.
ICAO J, 51 (1996), pp. 11-13
P. McCulloch, J.A. Cook, D.G. Altman, C. Heneghan, M.K. Diener.
IDEAL framework for surgical innovation 1: the idea and development stages.
BMJ, 346 (2013), pp. f3012
P.L. Ergina, J.S. Barkun, P. McCulloch, J.A. Cook, D.G. Altman.
IDEAL framework for surgical innovation 2: observational studies in the exploration and assessment stages.
BMJ, 346 (2013), pp. f3011
J.A. Cook, P. McCulloch, J.M. Blazeby, D.J. Beard, D. Marinac-Dabic, A. Sedrakyan.
IDEAL framework for surgical innovation 3: randomised controlled trials in the assessment stage and evaluations in the long term study stage.
BMJ, 346 (2013), pp. f2820

Please cite this article as: McCulloch P. La cirugía como disciplina científica y la IDEAL Collaboration. Cir Esp. 2014;92:71–73.

Copyright © 2013. AEC
Article options
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos

es en pt
Política de cookies Cookies policy Política de cookies
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here. Utilizamos cookies próprios e de terceiros para melhorar nossos serviços e mostrar publicidade relacionada às suas preferências, analisando seus hábitos de navegação. Se continuar a navegar, consideramos que aceita o seu uso. Você pode alterar a configuração ou obter mais informações aqui.