Translation into Spanish and validation of feedback in medical education questionnaire (FEEDME-Culture) during clinical rotations

Resumen

Introducción

El cuestionario Cultura de retroalimentación en educación médica es una escala validad que determina la percepción de la cultura de retroalimentación en países de habla inglesa. El objetivo de este trabajo es validar este cuestionario en su versión española en estudiantes de medicina peruanos durante una rotación clínica.

Métodos

Un estudio observacional, analítico de tipo instrumental fue realizado en 139 estudiantes de medicina. La validez de la escala fue determinada por V-de-Aiken e intervalos de confianza. Se realizaron pruebas de análisis factorial y alfa de Cronbach.

Resultados

Todos los valores fueron estadísticamente significativos cuando se evaluaron con el coeficiente V-de-Aiken. Además, la asimetría y curtosis de todos los ítems son adecuados ya que no exceden el rango >± 1.5. La significancia de la prueba de esfericidad de Barlett (1521.8; gl = 120; p = 0.001) y el coeficiente Kaiser Meyer-Olkin (KMO = .953) fueron adecuados y el análisis factorial exploratorio reveló una estructura de dos factores. El coeficiente α de Cronbach de la escala total y sus factores, fue superior a 0,80.

Conclusiones

El cuestionario Cultura de retroalimentación en educación médica en español es una escala válida para evaluar la percepción sobre la cultura de retroalimentación en estudiantes de medicina que están realizando rotación clínica.

Abstract

Introduction

The Feedback-Culture in Medical Education questionnaire is a validated scale to determine students’ perceptions of feedback culture in English-speaking countries. The objective of this work is to validate this questionnaire in its Spanish language version for Peruvian medical students during a clinical rotation.

Methods

An observational, instrumental-type analytical study was conducted on 139 medical students. The validity of the scale was determined by means of the Aiken's V coefficient and confidence intervals. Factorial analysis tests and Cronbach's alpha were performed.

Results

All values were statistically significant when evaluated with Aiken's V coefficient. In addition, the asymmetry and kurtosis of all items are adequate as they do not exceed the range >±1.5. The significance of the Barlett sphericity test (1521.8; gl = 120; p = 0.001) and the Kaiser Meyer-Olkin coefficient (KMO = .953) were adequate and exploratory factor analysis revealed a two-factor structure. Cronbach's α coefficient of the total scale and its factors was higher than 0.80.

Conclusions

The Feedback-Culture in Medical Education – Spanish questionnaire is a valid scale to evaluate the perception about feedback culture in medical students who are doing a clinical rotation.

Introduction

Medical education is currently developing with an important focus on competence-based education, where feedback for students is one of the essential elements in the development of competencies.^1,2 In the clinical learning environment, receiving feedback also significantly influences the acquisition of new skills, knowledge, attitudes and professional development.³ Structurally integrating feedback into the medical school curriculum can influence the development of a culture of trust and should promote students’ long-term personal and professional development.⁴

Thus, feedback is considered crucial to guarantee the adequate teaching-learning process in the field of medicine.⁵ However, many students consider that they do not receive the feedback they need, particularly during clinical rotations.⁶ Few instruments have been designed worldwide to measure the feedback culture in medical students, all of them in the English language,^7,8 among which is the Feedback in Medical Education (FEEDME-Culture) questionnaire, which measures the perception that medical students have of the feedback culture during clinical rotations.⁸

Currently, there is no instrument in Spanish that is useful for measuring the feedback culture in medical students through which the delivery and reception of feedback between teachers and students can be improved. The objective of this study was to translate, adapt and validate the FEEDME-Culture questionnaire in its Spanish language version for medical students during a clinical rotation.

Methodology

Design and sample

This study employed an observational, cross-sectional and instrumental design. The study was conducted with medical students from third to sixth academic year, who were actively participating in a practicum-based part of their degree program in both public and private healthcare facilities; 195 students were in these academic periods performing their clinical rotations in some of the following services: rheumatology, dermatology, neurology, endocrinology, ophthalmology, neoplasms, gynecology, psychiatry, urology and obstetrics. Each clinical rotation lasted three weeks. For the pilot study the sample group was composed of 139 students, of which 57 were male and 82 females with 21.29 years old average (SD = 3.911). The sample size coincided with the original study for the English version of the instrument.

Procedure

This study was divided into translation and validation stages, focused on the FEEDME-Culture questionnaire.⁸ The author of the original English version was contacted before starting this study, and he gave his permission to translate the instrument and was consulted throughout the process. The study was reviewed and approved by the Ethics Committee from Universidad Peruana Union.

The translation process was carried out according to Guillemin, et al.⁹ Four native South American Spanish speakers who are fluent in English prepared their own translations and then met to discuss differences and consolidate into one preliminary Spanish version. Then two certified public translators with no knowledge of the project translated this initial version from Spanish back into English, and the translation team prepared a consolidated back-translation, which was sent on to the original author for his review. Finally, a focus group consisting of one classroom of medicine students gave their comments about the instrument and the final adjustments were made before applying the pilot study. Appendix 1 shows the version of the instrument which was used in the pilot study, which includes the changes made due to comments from the focus group and expert review, compared with the original English version.

The validation stage consisted in a revision by experts and a pilot study. The experts review focused on content validity including relevance, representativeness and clarity of the items. Seven medical professionals who teach in higher education were the judges, and the Aiken's V value was calculated.

For the pilot study, two of the authors presented the project in a regularly scheduled meeting of medical students, held on the campus of the Universidad Peruana Unión, after the completion of the clinical rotations corresponding to the academic period. There the students were asked to participate, the voluntary nature of their participation was explained, and a QR code and short link were shown so that the students could participate via their cell phones and laptops. Both options linked to the same Google form, which included an initial page with informed consent and then continued by asking for the student's ID number (to avoid repeat completion) and gender before continuing to the instrument itself.

Statistical analysis

The descriptive statistics of the scale were analyzed, then an exploratory factorial analysis (AFE) was carried out, this by means of the unweighted least squares method, with prominent oblique rotation and based on a Pearson's correlation matrix. The Kaiser–Meyer–Olkin measure and the Bartlett sphericity test were performed. The program FACTOR Analysis version 10.1 and SPSS software version 25.0 were used.

Results

Expert judgment was requested to analyze the relevance, representativeness and clarity of the items in the FEEDME-Culture–Spanish questionnaire (Appendix 2). All the items received a favorable evaluation by the experts. Regarding relevance, items 4, 5, 8, 9 and 12 are the most important. The item 4 was found to be the most representative, and items 2, 3, 4, 8 and 14 were better evaluated regarding clarity.

Appendix 3 shows the descriptive statistics of the 16 items of the questionnaire. The item 4 has the highest average score (M = 3.662) and item 2 (SD = 1.302) shows the greatest dispersion. The asymmetry and kurtosis of all items in the FEEDME-Culture-Spanish are adequate, as they do not exceed the range >±1.5.

The significance of the Barlett's sphericity test (1521.8; gl = 120; p = 0.001) and the Kaiser–Meyer–Olkin coefficient (KMO = 0.953) were adequate. The parallel analysis indicated the existence of two factors (Table 1). All items presented saturations higher than 0.3. However, items 1, 2, 8, 9 and 12 were eliminated, because they do not theoretically correspond to their factor.

Table 1.

Exploratory factor analysis of the FEEDME-Culture scale in Spanish.

Items	F1	F2
3. El feedback que recibí contenía detalles específicos sobre mi desempeño.	0.544
4. El feedback me llevó a reflexionar sobre mi desempeño.	1.000
5. El feedback incluyó sugerencias para ayudarme a mejorar.	0.739
6. Cuando recibí el feedback sobre cómo mejorar, sentí que las expectativas eran razonables y factibles.	0.727
10. El feedback ayudo a mejorar mi desempeño.	0.514
11. El feedback que recibí en esta rotación ayudó a identificar mis fortalezas y debilidades.	0.462
13. Entendí el propósito del feedback que recibí durante esta rotación.	0.324
7. Se separó un tiempo exclusivo para darme el feedback.		0.764
14. Recibí feedback de reafirmación (lo que debería seguir haciendo) y corrección (lo que necesito mejorar).		0.991
15. Hubo seguimiento del feedback que recibí con el fin de revisar mi progreso.		0.971
16. La cantidad de feedback que recibí cumplió mis necesidades de aprendizaje.		0.851

The first factor explains 72.90% and factor 2 provides 7.02% of the variance of the construct. The correlation between the factors was significant >0.50. Robust analyses (CFI = 1.00, GFI = 0.992, TLI = 1.00 and AGFI = 0.997), show that the factor structure is satisfactory. Regarding the reliability of the total scale and its factors, Cronbach's α coefficient is higher than 0.80 (Table 2).

Table 2.

Correlations between factors and reliability of the FEEDME-Culture scale in Spanish.

Variable	F1	F2	% EV	α	95% CI
Factor 1	1		72.90	0.94	0.91–0.95
Factor 2	0.869*	1	7.02	0.92	0.89–0.94
Total			80.0	0.90	0.86–0.92

Note: EV = explained variance; α = Cronbach's coefficient; CI = confidence intervals.

*

p < 0.05.

Table 3 shows the descriptive statistics of the 11 items of the FEEDME-Culture-Spanish questionnaire and reports communities >0.60, the correlations are moderate and significant between items (>0.70).

Table 3.

Descriptive statistics and reliability of the 11 items of the final FEEDME-Culture scale in Spanish.

Variable	M	SD	As	K	h	r itc	α
Factor 1
Item 1	3.094	1.038	−0.189	−0.541	0.704	0.825	0.958
Item 2	3.662	1.141	−0.623	−0.414	0.695	0.713	0.962
Item 3	3.554	1.138	−0.590	−0.363	0.750	0.818	0.958
Item 4	3.504	1.042	−0.664	0.180	0.767	0.815	0.959
Item 6	3.583	1.072	−0.587	−0.159	0.808	0.880	0.956
Item 7	3.612	1.083	−0.581	−0.182	0.783	0.866	0.957
Item 8	3.655	1.104	−0.801	0.109	0.754	0.853	0.957
Factor 2
Item 5	3.309	1.210	−0.393	−0.686	0.653	0.782	0.960
Item 9	3.302	1.209	−0.402	−0.785	0.826	0.833	0.958
Item 10	3.029	1.199	−0.182	−0.898	0.717	0.759	0.960
Item 11	3.309	1.161	−0.514	−0.466	0.846	0.875	0.956

M = mean; SD = standard deviation; As = asymmetry coefficient; K = kurtosis coefficient; h = communities.

Discussion

Feedback can be vital for improving effective exchange of feedback in the clinical learning environment.¹⁰ We translated into Spanish and validated the FEEDME-Culture questionnaire.⁸ The original author was fortunately willing to help with the project. It was hard to translate the core concept of this instrument, feedback, and the focus group was able to confirm the translation team's impression that the term is common enough as a loan word from English that it could be used without translating it, so as to not lose part of the concept in the translation. The translator team helped not only translate the instrument but also to incorporate suggestions by the original author, the focus group and the expert judges. This helped ensure a more accurate and understandable translation.

With respect to content based validity, the experts in medical education gave their opinion which was quantified through Aiken's V finding that the items are clear, representative and relevant to measure feedback culture, this finding is similar to what was reported in the original instrument where the Delphi method was used.⁸

An AFE was applied and unlike the original scale, the unweighted least squares method was used. The parallel analysis suggested a 2-factor model, being consistent with the original structure: “feedback characteristics” and “feedback delivery”. The findings of the study show that the FEEDME-Culture-Spanish questionnaire in the studied sample is configured by 11 questions, this is due to the fact that five items do not saturate the original factor (theoretical rationality).¹¹

The first factor analyzes the impact of feedback on student performance during their clinical practice. This activity can improve students’ learning and academic development more quickly than conventional methods of education.^12–14 On the other hand, feedback can be extremely useful when making curricular changes or trying new learning methods in a medical school.¹⁵ Students trying out simulated patients after receiving feedback from their instructor, proved to be a much more effective method than conventional education.¹⁶ In addition, the use of feedback can not only be applicable among students, but can improve the skills development at different stages of the physician's educational development.¹⁷

The second factor inquires about students’ perceptions and willingness to give feedback. One of the items delves into the students’ willingness to set aside some time from their practice for the exclusive development of this activity. In some cases, both students and teachers are not willing to allocate time for feedback because they do not know the appropriate methods for giving feedback.¹⁸ Learning these feedback methods can improve stress and pressure including highly demanding jobs.¹⁹ This factor also concerns the effect of correct feedback on the evaluation of the participant's progress, this has been proven in follow-up studies in virtual classes, where feedback helped to determine learning and knowledge evolution in students of different educational levels and can become crucial when the student self-manages his learning.^14,20

The internal consistency of the FEEDME-Culture-Spanish questionnaire is good (α = 0.90 [95% CI 0.86–0.92]) demonstrating that the instrument is reliable, similarly to the original instrument where Cronbach's alpha values for the full scale and its dimensions were reported to be between 0.89 and 0.91.⁸ It should be noted that in the original study the CIs are not reported, considering that the CIs are necessary for a correct interpretation of the metric properties and their practical usefulness.²¹

The limitations of the research: the size of the sample, medical students from one university participated, confirmatory factor analysis was not applied, since there was no other instrument in Spanish that measures feedback culture from medical students, criterion validity was not applied.

Despite the limitations, we consider the FEEDME-Culture-Spanish questionnaire is valid and reliable for the evaluation of culture feedback in medical students with similar contexts to Peru, such as South American countries.

Authors’ contributions

MW and SHV were involved in conception, study design, interpretation of data, drafting and revising of manuscript. JdFC, MDCA were involved in acquisition, interpretation of data and drafting of manuscript. RFCE, JFRA and CRM were involved in analysis and interpretation of data, drafting and revising of manuscript. All authors read and approved the final manuscript.

Conflict of interest

The authors declare that they have no conflict of interest.