Most prospective, controlled, randomized studies (PCRS) are so-called superiority studies, which aim to determine whether one intervention is superior to another. The fact that no statistically significant differences are found between two interventions does not mean that they are similar or equivalent. For this reason, there are equivalence trials, which aim to determine whether a new intervention is therapeutically similar to another existing one.1
Non-inferiority studies (NISs) assess whether a new treatment is not considered worse, or inferior, compared to the regular treatment. These studies provide an answer as to whether we are willing to accept a new intervention that may be clinically inferior, up to a certain point, but that is still beneficial for patients and has some additional advantage, such as a less aggressive, safer and more cost-effective treatment–efficiency.2 The original CONSORT (CONsolidated Standards of Reporting Trials) statement was developed to help address the problem of inadequate submission of PCRSs. To facilitate the development and quality of the NISs, an extension of the CONSORT 20101 guide was published.
Non-inferiority and equivalence are sometimes mistakenly used in the same way. Equivalence trials are defined as the treatment effect being within both non-inferiority margins (NIM), above and below (−Δ and +Δ).
The low quality of these studies can bias their results and produce false positives for non-inferiority.2 The objective of this article is to address the presentation form of NISs.
Main NIS characteristicsNISs must follow the checklist proposed by the CONSORT guide adapted for this type of study.1 A series of characteristics mark their identity and all of them must clearly reflect these to ensure the NIS quality:2
- 1
The non-inferiority margin (NIM). This is the value that allows a new treatment to be considered "acceptably worse."
- 2
The types of population group analysis must be defined.
- 3
The consistency of the confidence intervals (CI) must be defined.
- 4
Missing data management. Some missing data are inevitable, but in the NISs the missing data may be biased towards demonstrating inferiority.
Superiority PCRSs can be classified as positive or negative depending on whether the null hypothesis is rejected (i.e., p < .05). In contrast, the NISs can be interpreted as non-inferior, inconclusive, inferior or superior, depending on the location of the 95% CI.4
Fig. 1 describes all the possibilities of the results of an NIS depending on where the 95% CI is located. In the literature, the term non-inferiority has sometimes been used when the difference between two treatments is inconclusive without reference to a non-inferiority margin. This use is inappropriate. If the trial was underpowered, the actual difference could still be substantial.
Location of the possible results of the 95% confidence intervals. The solid line is 0, which indicates statistical significance when the entire interval is on one side or the other of 0. The dashed line is the one that indicates the non-inferiority margin (Δ= −10). The shaded area to the left indicates the non-inferiority area. (A) If the CI is completely to the left of zero, the new treatment is superior and of course, not inferior. (B) If the CI lies to the left of Δ and includes zero, the new treatment is not inferior but has not been shown to be superior. (C) If the CI is completely to the left of Δ and completely to the right of zero, the new treatment is not inferior in the sense already defined, but it is also inferior in the sense that a zero-treatment difference is excluded, so it will be statistically lower. This puzzling circumstance is rare because it requires a very large sample size. It may also be due to a non-inferiority margin that is too wide. (D) If the CI includes Δ and zero, the difference is not significant but the result on non-inferiority is inconclusive. (E) If the CI includes Δ and is completely to the right of zero, the difference is statistically significant but the result is inconclusive with respect to a possible inferiority of magnitude Δ or worse. (F) If the CI is completely to the right of Δ, the new treatment is inferior.1
In the NIS, creating a figure with the position of the 95% CI (Fig. 1) with the NIM helps readers interpret the result.1 Thus, as seen in Fig. 1, cases C, E and F were statistically significantly lower, but nevertheless in terms of a NIS, C was non-inferior, E inconclusive and F lower, in relation to the NIM.5
Non-inferiority can be assessed as either a relative or absolute difference. Both have their advantages and disadvantages. The absolute difference is often the most relevant to patient outcomes, as it reflects a change in reduction or improvement associated with standard surgical treatment. It can be more easily compared to other treatment effects.
The relative difference may be particularly useful in understanding how the new treatment compares to the conventional treatment in terms of the total effect of treatment versus no treatment.3
How to define the non-inferiority margin (Δ)Determining the NIM, also called delta (Δ), is the main challenge in designing an NIS. Choice is essential to reach appropriate conclusions. The NIM should depend on the importance of the final result and the magnitude of the new treatment.1
Despite being a key component in the design and interpretation of NISs, recommendations for the selection of NIM are unclear. A frequently used statement in some articles is, “the choice of margin when it is clinically acceptable.” This is not sufficiently consistent to justify the MNI. In general, the margin should be chosen on a clinical basis, incorporating statistical evidence from meta-analyses or other quality clinical studies.6 If the choice of margin is based on a group of clinical experts, information should be provided on how many experts were involved and how many considered acceptable the choice of margin (as in Delphi processes).2
How should the sample estimation be made?It differs considerably from superiority studies, as they are directly defined by the NIM (Δ). If Δ is too large, the risk of accepting both truly inferior and non-inferior treatment will be very high. This situation is especially relevant for important outcomes such as mortality. On the other hand, setting Δ as too small can produce inconclusive results, requiring an extremely large trial.
The value of the statistical power (1-β) must be known and this is assigned to the study. Most researchers assign a β = .2 value so the power is .8. This may be higher if it is assumed that the new treatment is more effective than the reference treatment or lower if it is assumed that it is less effective. Also, the type I risk alpha (α) must be related as specified in the non-inferiority hypothesis. The bilateral contrast type provides additional information, particularly for the situation where the new treatment is superior to the regular one.
Sample size calculation formulas for non-inferiority studies for a primary variable with two categories (2A) or a continuous variable (2B) are shown in Fig. 2. In the following links the online calculators for NISs with both types of variables may be accessed:
https://www.sealedenvelope.com/power/continuous-noninferior/7 and https://www.sealedenvelope.com/power/binary-noninferior.8
Specify the type of population group analysis that is being assessedBoth intention-to-treat (ITT) and per-protocol analyses should be considered to meet non-inferiority. Generally, the ITT analysis population is the most appropriate, but as previously mentioned, this may underestimate the real difference between treatments, and it is important to also describe compliance with treatment in both groups.3
Should an interim analysis be performed? Study detentionIn superiority trials, when an interim analysis shows clear evidence of the efficacy of the new treatment, it may be considered ethically incorrect to continue the trial and deny the effective new treatment to the control group. In contrast, in an NIS, if non-inferiority is demonstrated for the primary outcome before completion of the trial, there is less ethical need to stop the trial because the control group is already receiving standard treatment. It is therefore suggested to finalise the NIS and be able to specifically define where the CI is located.
Final commentThe main characteristics of the NISs that we have described must be clearly described. Any absence in their definition can lead to false conclusions.2 This is particularly the case of those NISs with inconclusive results, which can misinterpret the true results of the trial.
