How do you know if you should perform a meta-analysis in a field where previous meta-analyses have already been performed?

We present several arguments that would justify a new meta-analysis (MA) when another one that addresses the same question has already been published (assuming that the question that motivated the first MA is still of interest):

• 1

  Update due to new studies. The mere existence of new studies can be an argument, even if the previous MA is not questioned. Since a MA can serve as a basis for planning future research, the broader the information base of the MA, the more appropriate such planning will be.¹ But it also does not seem reasonable to update it with each new study. Authors, reviewers, and journal editorial boards must have a way of deciding whether merely increasing the study base justifies a publication that will be redundant in methods and probably in the main conclusions. A first basis for the decision may be to increase the power of the statistical tests and therefore the validity of the statistical conclusion. Many times meta-analytic tests do not reach an adequate or desirable level of power. Suppose that after calculating the a posteriori power of the test of the null hypothesis on the average effect (or the single effect) it turns out to be too low. Then, it can be calculated the number of studies of size equal to the average size of those already published to reach an adequate, or desirable power.² If the studies published after the last MA reach such a number, there is already a well-founded justification for the update. Even if the ideal power is not reached, a substantial increase in power may be a sufficient reason for upgrading. A second basis may be the improvement in the precision of the estimates. Applying the same logic as with power, the number of new studies necessary to reduce the width of the interval to a certain magnitude can be calculated. Furthermore, both criteria can be applied to global estimates and tests as well as to moderator analysis or to the estimation of heterogeneity.

• 2

  Critically correct the previous MA. Sometimes some decisions of the published MA are not shared, because they are considered incorrect or because important issues are omitted. The reasons for the discrepancies and what is expected to be found when modifying the decisions and criteria should be clearly explained. These reformulations are often a consequence of the criticism and debates generated with the publication of the original MA.³ Under this motivation, the authors must make clear what they want to modify from the previous meta-analysis and why. Third-level studies, such as overviews of reviews or umbrella reviews.⁴ can help identify weaknesses and gaps in published MAs and thus improve the design and planning of updates.

• 3

  Deep changes in the field. A very typical example is the identification of new moderating variables, because information has begun to be collected regarding variables that were not previously reported because they were not considered relevant. Likewise, it may have been discovered that it is advisable to disaggregate some of a moderator's categories into two or more categories that until then were treated together. This offers new opportunities for analysis that can help qualify conclusions. Another example, when controlled trials are synthesized, is a change in the “treatment-as-usual” or default comparison. The new effect size estimates cannot be directly averaged with the old ones. Updates are especially indicated when new treatments have been introduced and the objective is to expand the structure of a network meta-analysis.⁵

• 4

  Technical and methodological innovations in MA. Sometimes new developments occur in meta-analytic methods, which replace previous ones or which are applied to new questions. For example, in the last two decades, detection and correction of publication bias or heterogeneity estimation methods have been greatly developed. Likewise, new tools such as Risk of Bias⁶ have been developed. In previous MAs others will have been used that, although they were the most appropriate at the time, are no longer suitable according to current standards. Reanalyzing a meta-analysis, even if few or no studies are added, may be justified if substantial modifications are introduced in the techniques and procedures that clearly improve the reliability and validity of the estimates and conclusions.

The above motivations are neither exhaustive nor mutually exclusive. It is not difficult to find new MAs in which new studies are added compared to a previous one, but some of its decisions are also modified (justifiably) and new moderators that were not previously considered are analyzed. All of them can be used, both by the authors when justifying their proposals and by the reviewers and editors to accept them. The latter must be vigilant against spurious motivations, more related to the authors' careers and their need to publish than to the advancement of knowledge.⁷ In any case, whether there are previous MAs or not, and whatever the reason, every MA must explicitly indicate the justification that motivates it.⁸ If a previous one exists, its results and main conclusions should be summarized, something that is often not done.⁹