In current surgical practice, we are continually making clinical-therapeutic decisions regarding our patients. At surgical conferences, however, it is striking to see how the clinical-therapeutic management proposed for a given case can vary greatly among different surgical groups with recognized experience. Furthermore, when scientific information is sought to possibly reach a consensus on these issues, it is not uncommon to find publications in high-impact journals with relatively contradictory messages.
It is therefore important to know how to obtain the best scientific evidence to treat each of our patients. Traditionally, we have relied on statistical significance (P-value) to evaluate results in clinical research,1 but a statistically significant result might not be clinically significant, and vice versa.2
To assist in making correct clinical decisions, Evidence-Based Medicine3 has emerged as a professional practice that integrates clinical experience, the specific characteristics of each patient, and the best scientific evidence available. However, it is not easy to incorporate this evidence into professional practice given the large number of scientific articles published each year, which in surgery exceeds 45 000 articles annually in indexed journals. Not only is the volume of scientific production problematic (especially as it increases exponentially year after year), but there are also issues of methodological variability and quality of the research, as well as the risk of bias in the results of many studies. Consequently, the results of different studies may be contradictory, even within the same area of surgery.
For all these reasons, we surgeons need to acquire sufficient methodological knowledge to be able to identify the best scientific evidence. Nevertheless, we must be aware that our ability to select the best evidence for our surgical activity will be influenced by the volume of publications, the need to adequately evaluate their methodologies, and our usual lack of time to adequately analyze and consider the reliability of the results.4
In the search for solutions to simplify this situation and enable us to conduct our surgical practice with guarantees, clinical practice guidelines have been developed based on the selection of the highest quality scientific articles. In this context, the best instruments for the synthesis of scientific literature are Systematic Reviews (SR) and Meta-Analyses (MA). These instruments compile all studies on a specific clinical question and review of their methodologies, study populations, results, and risk of bias, while using a strict, replicable and precise methodology.5 In addition, MA allow for quantitative analysis of the results. For the above reasons, SR and MA are the foundation used to apply scientific evidence to professional practice. They are considered the highest level of evidence for the development of recommendations in clinical guidelines.
However, SR and MA also present biases and limitations. Their intrinsic quality is therefore crucial for the validity and reliability of their conclusions. Thus, it is necessary to correctly carry out each of the stages of the SR and MA, and we must understand the specific tools that enable us to evaluate the quality of the methodology (eg, AMSTAR-2) and the risk of bias (eg, ROBIS).6 Furthermore, the presentation of the results also influences the quality of the SR and MA, since they must include minimum items that guarantee reliable results, as indicated by the PRISMA guidelines for randomized clinical trials7 or the MOOSE guidelines for observational studies.8 In short, the factors that mainly affect the quality of SR and MA are those related to the original studies and the development of the SR or the MA itself.
Several authors have demonstrated deficiencies in MA that have had an impact on their quality.6 Frequently, one of the limiting factors specific to SR and MA in surgery is the incomplete recording of data related to the intervention. Authors like Yu et al9 have indicated that proper descriptions are only found in 40% of the studies included, and this deficiency is also observed in studies on postoperative complications.10 Another quite specific aspect of surgery is the performance of MA with few studies. More than 35% of MA published in surgery include 10 or fewer publications,11 which is accentuated by the tendency among surgeons to perform observational studies instead of randomized clinical trials, a limiting factor of quality.12
Consequently, it is important to have a minimal understanding of these types of studies, and surgeons must know how to interpret the quality of SR and/or MA before using them in our decision-making process13 in order to provide quality care in our clinical practice. This is especially true nowadays, when the publication of SR and MA is proliferating and even exceeds the publication of primary studies in certain fields.14 We should also remember that, although instruments are available to analyze their quality, they are sometimes not applied rigorously, which is favored by an academic and professional system where publishing at a certain speed is encouraged. Therefore, surgeons must have interpretative and critical reading skills to analyze SR and/or MA, which entail rigorous critical analysis of the validity of the article, interpretation of the results, and comprehension of their possible relevance.15 The critical review of scientific manuscripts, in this case SR and/or MA, continues to be a priority in all areas of medicine, especially in surgery.
Finally, it should be noted that neither the publication in a high-impact journal, nor a review by peers, nor the compliance with PRISMA guidelines when conducting SR and/or MA, nor even the completion of the study by a highly prestigious organization like the Cochrane Collaboration ensures the absence of errors in scientific manuscripts.16 It therefore remains the surgeon’s obligation to determine, through an easy and standardized process, whether or not the results derived from the research should be transferred to the patients being treated.
In this context, the Cirugía Española journal will become involved in this training by presenting a series of Methodological Letters to review the fundamental aspects of SR and MA.
