There are three main objectives in the statistical analysis performed in a meta-analysis: (a) estimate the population mean effect by calculating the (weighted) average effect and a confidence interval around it; (b) check whether the effect sizes of the synthesized studies exhibit more heterogeneity than can be explained by mere sampling error; and (c) if heterogeneity is observed among the effect sizes, investigate what characteristics (moderators) of the studies can explain this variability.1,2
Meta-analysis offers a good opportunity to analyze study characteristics as potentially moderating variables of the variability of effect sizes. The heterogeneity exhibited by the effect sizes may be due to substantive or methodological characteristics.3 The substantive characteristics have to do with the objective of the studies, such as the type of surgery, the severity of the patients, their average age or their distribution by gender. Methodological characteristics refer to the methodology of the study. It is worth mentioning the type of design (cohort, case-control, cross-sectional), experimental mortality, and a large number of characteristics related with the methodological quality of the study.
When the moderating variable is quantitative in nature (e.g., experimental mortality, year of publication, average age of the sample), a meta-regression model is applied. Meta-regression is a linear model that employ some weighting procedure based on the assumed statistical model, which is generally a mixed effects model, an extension of the random effects model that analyze the influence of potential moderators.4 When the moderator variable is categorical (e.g., type of design, country of the study, surgical technique), a technique that in meta-analysis is called “subgroup analysis” is applied. It is equivalent to a weighted analysis of variance (ANOVA), also assuming a mixed effects model.5,6,7
In the meta-regression case, the statistical analysis consists of applying a statistical test that allows us to assess whether the moderator is statistically related to the effect sizes. This statistic can be complemented with the calculation of the proportion of the variance explained by the moderator. If the statistical test reaches statistical significance, it is important to interpret the sign (positive or negative) of the regression coefficient to highlight the direction of the relationship between the moderator and the effect sizes: a direct relationship, if the regression coefficient is positive, or an inverse relationship, in the opposite case.4
In the case of subgroup analysis, the effect sizes of the studies are grouped according to the categorical moderator and the average effect size of each category of the moderator is obtained, together with its confidence interval and the heterogeneity statistics (Q, I2, etc.). Likewise, a statistical test is applied to determine if the mean effect sizes of the moderator categories are statistically different. These analyzes are also complemented by the calculation of the proportion of the variance explained by the moderator.5,6,7
An example is the meta-analysis by Ricci et al.8 on the effectiveness of the Blumgart anastomosis (BA) versus its non-application during duodenal-pancreatectomy. The effect size they employed was the Risk Difference between BA and non-BA of suffering clinically relevant pancreatic fistula. One of the moderators they analyzed was the quality of the studies using the Methodological Index for Non-Randomized Studies (MINORS). The simple meta-regression revealed the existence of a statistically significant negative relationship between this index and the risk differences (bj = −0.02, p = .022, R2 = 46%): studies with lower methodological quality tended to offer results more favorable to BA.
It is also possible to apply multiple meta-regression models, in which the influence of several moderators is tested simultaneously, in order to propose a predictive model of the variability of effect sizes incorporating several moderators simultaneously.4 To control the phenomenon of capitalization of chance, it is advisable to apply the 1:10 ratio, according to which no more than one moderator should be included in the meta-regression model for every 10 studies in the meta-analysis.1
When interpreting the results of the moderator analysis, it is important to distinguish between two types of moderators: moderators of the study and moderators of the participants (patients).3 Study moderators are unique characteristics of each study, such as the type of surgery performed in the study, the type of design, or the method of concealment of patient randomization. The moderators of the participants refer to their individual characteristics, such as the age or gender of the patients. Since the patients are nested in the studies, these characteristics have to be represented through numerical summaries (e.g., the average age of the sample or the percentage of males). This circumstance makes the analysis of patient-related moderators less powerful, leading to less solid conclusions about the relationship between those moderators and effect sizes than when study moderators are analyzed.
