Nursing research is essential for improving both the quality of the nursing team management and the delivery of care. Validated and reliable questionnaires, surveys, or measurement instruments are frequently used for this purpose. In clinical nursing, it is essential to have precise instruments that guarantee what is intended to be measured.1

The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) taxonomy considers content validity to be the most important measurement property and recommends assessing content validity using proposed standards for relevance, comprehensiveness, and comprehensibility. It offers a checklist to ensure a systematic and transparent evaluation of the content validation of measurement instruments.2

There are content validation methods that involve one or two rounds of expert judgment and statistical analysis. Expert judgment analysis involves selecting a panel of experts in the subject area who assess the relevance and representativeness of each item in the instrument. It consists of the following stages: 1. Expert selection: Experts must have in-depth knowledge and experience in the instrument's subject area; 2. Item evaluation: Each expert reviews the instrument's items and rates their relevance using a scale (e.g., 1 to 4, where 1 is “not relevant” and 4 is “very relevant”); 3. Results analysis: The level of agreement among experts is calculated, for example, by averaging the relevance ratings for each item, and determining whether the items meet the predefined threshold of acceptability.

In nursing, various instruments are used to assess aspects such as quality of life and patient safety. Given the widespread use of measurement instruments in clinical nursing, the importance of content validity, and the lack of similar previous studies, it was considered relevant to compare different techniques for studying content validity using a practical case study, with the aim of providing rigour to nursing care research. In this context, the objective of this study was to compare different techniques for analysing the content validity of measurement instruments applicable to nursing care research using a practical case study.

A secondary study derived from the validation of a hospital-based survey on patient safety (HSOPS) in a Chilean hospital.3 The analysis included three phases: 1) identification of the main methods for assessing content validity; 2) calculation according to each method; and 3) differences and similarities between the methods.

The setting was a high-complexity hospital in Valparaíso, Chile, based on primary research conducted in 2021.3 The population consisted of 12 experts: six academic nurses with a master's degree and five years of teaching experience in healthcare management or research, and six nurses with five years of experience in the hospital's quality management and patient safety units.

The variables corresponded to the level of sufficiency, clarity, coherence, and relevance assigned by each expert for each item. A score was obtained based on a Likert scale ranging from 1 to 4, with 1 being irrelevant and 4 being extremely relevant.

Data collection was carried out by email to each expert. Two rounds were conducted to achieve ACCEPTABLE levels of content validity. The five techniques compared to identify differences and similarities were Lawshe,4 Lawshe & Tristán-López,5 Lynn,6 Polit & Beck,7 and Hernández Nieto,8 as described in Table 1. These techniques are primarily proportional, based on the sum of experts who rated the item positively over the total number of experts. Lawshe proposes an index (similar to a correlation) that ranges from −1 to 1. On a three-value scale, the calculation considers the number of experts who rate the item as essential.

Lawshe, modified by Tristán, simplifies the previous method by using the proportion of the total number of experts who rated an item as essential over the total number of experts. This makes the method easier to interpret, ranging from 0 to 1.

Lynn estimates the content validity index (I-CVI) and measures the proportion of experts who rate an item as “fairly or extremely relevant” out of the total number of experts. This is very similar to the previous method, except that the scale used is a 4-value scale, not a 3-value scale.

Polit and Beck propose a correction to the previous index with the modified Kappa, calculating the probability of agreement by chance to subtract it from the I-CVI value, thereby ensuring the reduction of any statistical distortion.

Hernandez Nieto measures the ratio obtained between the mean scores assigned to the item and the maximum score that can be obtained as a rating for that item. This is interpreted as the “level of achievement” achieved. It also corrects for the statistical error assigned to each item, which is a constant. When multiple dimensions are assessed, the calculation is performed by adding together the four dimensions for each item and not individually for each dimension. Additionally, the Kruskal-Wallis test was applied to evaluate overall differences between the method distributions, and Dunn's post hoc test was used to identify method pairs with significant differences.

The results obtained for the “relevance” characteristic for each item and the average for each dimension of the HSOPS 2.0 instrument during the cross-cultural adaptation process are presented in Table 2.

Similarity is observed between the Lawsche, Lawsche-Tristan, Lynn, and Polit & Beck methods, as 81.25% of the items (n = 21) obtained values ​​equal to 1. The Lawshe and Lawshe-Tristan values ​​are practically the same, but on a different scale. The Lawsche-Tristan and Lynn values ​​are the same due to the way the data were grouped to be consistent with the Lawsche-Tristan three-value scale. In the case presented, values ​​3 and 4 of the scale were combined. This procedure is the same as that used by Lynn's method to calculate the I-CVI. The modified Kappa value was found to be similar to the I-CVI value, with a slight downward shift, which occurs as the value obtained decreases due to the probability of agreement with chance.

The Kruskal-Wallis test followed by Dunn's post hoc test identified that the CVC variable has significantly different distributions compared to CVR, CVR', I-CVI, and K* (p < .001).

In none of the methods applied were items eliminated when they obtained values ​​below the acceptance threshold defined for each one (see Table 1). The supplementary material details the calculation and characteristics such as adequacy, clarity, and consistency.

Using a practical case, five techniques for analysing the content validity of measurement instruments applicable to nursing care research were compared. It is important to identify the most appropriate way to calculate the content validity index, depending on the problem being addressed and the type of instrument being validated.9 The values ​​obtained were high, and there were no significant differences between the methods applied, due to the instrument having undergone a prior validation process which facilitated item selection even though it was in another language. The selection of experts was essential, including determining the criteria for their selection; their quantity; the rating process that includes a reminder, and the estimated timeframe for this.10

The first four techniques were similar, and the most optimal was Polit & Beck, due to its accurate collection of information. The differences in the I-CVI values ​​could be due to the nature of the calculation, since none of them received the maximum score. The items that did not obtain an “Excellent” rating coincide with low ratings from other methods, which could be due to the fact that they used characteristics other than relevance. In this sense, the Hernandez-Nieto method (CVC) presented greater differences and provided complementary information for the analysis. This difference was confirmed to be significant through statistical analysis using the Kruskal-Wallis test followed by Dunn's post hoc test.

This study is not without methodological limitations, such as using a single measurement instrument, with a narrow range of experts, and in a specific geographical context. Furthermore, the topic was approached from a case study rather than synthetic data. These limitations should be taken into account in future research. However, we believe that this study provides relevant results on different techniques for studying content validity.

In conclusion, the use of the Polit & Beck method for content validity in measurement instruments for clinical nursing research is recommended because it is more mathematically rigorous and more highly justified, providing solid support for research in care. Additionally, the Hernandez-Nieto method is recommended when validating more than one characteristic.

This research did not receive any specific support from the public, private, commercial, or non-profit sectors.