Buscar en
Educación Médica
Toda la web
Inicio Educación Médica The role of virtual interactive simulators in medical education: Exploring their...
Información de la revista
Vol. 22. Núm. 6.
Páginas 325-329 (Noviembre - Diciembre 2021)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
264
Vol. 22. Núm. 6.
Páginas 325-329 (Noviembre - Diciembre 2021)
Original article
Open Access
The role of virtual interactive simulators in medical education: Exploring their integration as an assessment methodology in clinical years
El papel de los simuladores virtuales interactivos en la educación médica: Explorando su integración como metodología de evaluación en los años clínicos
Visitas
...
Jaime Miguel Abreua,
Autor para correspondencia
a39030@fcsaude.ubi.pt

Corresponding author.
, Bruno Guimarãesb,c,d,e, Miguel Castelo-Brancoa,f,g,h
a FCS-UBI, Faculdade de Ciências da Saúde, Universidade da Beira Interior, Covilhã, Portugal
b Departamento de Medicina Física e de Reabilitação, Centro Hospitalar Entre o Douro e Vouga (CHEDV), Santa Maria da Feira, Portugal
c Departamento de Cirurgia e Fisiologia – Unidade de Fisiologia, Faculdade de Medicina, Universidade do Porto, Porto, Portugal
d Departamento de Saúde Pública, Ciências Forenses e Educação Médica – Unidade de Educação e Simulação Médica, Faculdade de Medicina, Universidade do Porto, Porto, Portugal
e Unidade de Investigação Cardiovascular, Faculdade de Medicina, Universidade do Porto, Porto, Portugal
f CICS-UBI, Centro de Investigação em Ciências da Saúde, Universidade da Beira Interior, Covilhã, Portugal
g C2ICB – Núcleo de Coordenação de Investigação Clínica das Beiras da Universidade da Beira Interior, Covilhã, Portugal
h CHUCB – Serviço de Medicina Intensiva do Centro Hospitalar Universitário Cova da Beira, Covilhã, Portugal
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Figuras (1)
Tablas (2)
Table 1. Parameters evaluated during the evaluation sessions, as they were by methodology and their weighting in the final score.
Table 2. Satisfaction questionnaire: the results are expressed as mean and standard deviation of an ordinal numeric variable between 1 (strongly disagree) to 7 (strongly agree).
Mostrar másMostrar menos
Abstract
Introduction

Assessment takes a fundamental role in medical education. In Portugal, the main assessment method used is multiple-choice questions (MCQ). The virtual interactive patients (VIP) as an assessment method can be indicated for the development of clinical reasoning for medical students of the clinical years. This study compared the use of MCQ and VIP as assessment methodologies in medical students.

Methods

A randomized crossover study carried out, which 35 Portuguese medical students from clinical years evaluated by MCQ and VIP in four medical specialties. In each specialty, students randomized into two groups. One group started being evaluated by MCQ and finished with VIP, the other vice-versa. After the evaluation, students asked to complete a satisfaction questionnaire.

Results

The final score was higher with the VIP than MCQ method (MCQ: 67.40±8.96 vs. VIP: 73.34±10.8; p<0.01) with significant differences higher in Clinical History and Physical Examination. The satisfaction quiz showed although both methodologies are adequate and satisfactory, but VIP method allows to improve knowledge.

Conclusion

VIP is more accurate for clinical reasoning, accepted by medical students and should be integrated into the pre-graduate medical curriculum with other methodologies already used. The COVID-19 pandemic allowed a digital transformation of medical education and VIP will have a crucial role as a knowledge and evaluation method.

Keywords:
Medical-students
Assessment
Simulation
Medical-education
Virtual-interactive-patient
Resumen
Introducción

La evaluación tiene un papel fundamental en la educación médica. En Portugal, el principal método de evaluación utilizado son las preguntas de opción múltiple (MCQ). El paciente virtual interactivo (VIP) como método de evaluación puede estar indicado para el desarrollo del razonamiento clínico para estudiantes de medicina en años clínicos. Este estudio comparó el uso de MCQ y VIP como metodologías de evaluación en estudiantes de medicina.

Métodos

Se realizó un estudio cruzado, aleatorizado, en el que 35 estudiantes de medicina portugueses de años clínicos fueron evaluados por MCQ y VIP en cuatro especialidades médicas. En cada especialidad, los estudiantes se asignaron al azar en dos grupos. Un grupo comenzó a ser evaluado por MCQ y terminó con VIP, el otro viceversa. Después de la evaluación, los estudiantes pidieron completar un cuestionario de satisfacción.

Resultados

La puntuación final fue mayor con el método VIP que con MCQ (MCQ: 67,40±8,96 vs. VIP: 73,34±10,8; p<0,01) con diferencias significativas mayores en historia clínica y exploración física. El cuestionario de satisfacción mostró que ambas metodologías son adecuadas y satisfactorias, pero el método VIP mejora el conocimiento.

Conclusión

VIP es más preciso para el razonamiento clínico, aceptado por los estudiantes de Medicina y debe integrarse en el plan de estudios de Medicina de pregrado con otras metodologías ya utilizadas. La pandemia por la COVID-19 permitió una transformación digital de la educación médica y VIP tendrá un papel crucial como método de conocimiento y evaluación.

Palabras clave:
Estudiantes de medicina
Evaluación
Simulación
Educación médica
Paciente virtual interactivo
Texto completo
Introduction

Assessment is one of the main moments in the learning process in medical education.1,2 Multiple-choice questions (MCQ) are the main assessment method used by Portuguese Medical Schools. The MCQ has the advantage of allowing the evaluation of high quantity of knowledge in a variable number of students and in a short time. However, good questions are difficult to elaborate without errors.3

Simulation in medical education has grown exponentially. Although it is typically used as a tool for teaching and evaluating the technical skills, interest has arisen in the possibility of using simulation as a method for knowledge assessment, namely, using virtual interactive patients (VIP), however, there is a lack of evidence.1,2,4–9 Studies suggested VIP could be a new method to evaluate medical students during their training course. Further studies are needed.1,6,7,9–14

Several factors have contributed to the decrease in the real contact of medical students with patients: limiting the exposure, reducing the time of patients in hospitals and the pressure on hospital funding. VIP has been recognized as an ideal and valid tool for medical education to teaching and assessment.7

VIP as a method of knowledge assessment has been suggested to be an efficient method with pedagogical value for the development of reasoning and clinical skills of medical students, given the possibility of completely managing a patient in a given clinical situation previously contextualized, where it is possible to collect a clinical history, perform a physical examination, prescribe complementary means of diagnosis and institute therapy. Thus, it is not limited to factual knowledge.7,10,15,16 This method has several advantages: efficiency, accessibility, interactivity and exploring rare and/or critical clinical cases. However, this method does not allow the assessment of cognitive skills (empathy, negotiation and communication of bad news).12 According to the opinion of medical students, this method has allowed an assessment of knowledge and clinical skills for clinical practice in the future.17

A direct comparison between the use of MCQ and VIP methods can be extremely difficult to apply since the use of VIP addresses skills and knowledge that are not present in MCQ. However, considering that the assessment also provides learning, the use of VIP can have positive implications both at the teaching and learning levels.2

In order to assess the impact of using VIP as a method of assessing knowledge in medical students from clinical years, this study was carried out, with the main objective of comparing the use of VIP and MCQ testing as assessment methods.

All medical students attending the clinical years (4th, 5th and 6th years) from all Portuguese Medical School were invited to voluntarily participate in 4 evaluation sessions of the present study. In each assessment session, each participant was assessed using 2 different methods: MCQ and VIP.

This study was approved by the Ethics Committee of the University of Beira Interior.

Methods

Similar studies have been developed.1,6,9,11 In the present study, 4 evaluation sessions were organized. Each session contained a different clinical case, from different medical specialties (Cardiology, Pulmonology, Gastroenterology and Neurology).

The clinical case presented was adapted so that it could be evaluated using the MCQ and VIP methods. The questions of the MCQ and the tasks to be performed in the VIP were grouped into five parameters and final score as shown in Table 1. All parameters and the final score were graded from 0 to 100%.

Table 1.

Parameters evaluated during the evaluation sessions, as they were by methodology and their weighting in the final score.

Parameters  MCQ  VIP  Final score 
Clinical history (Hx)  3–4 questions  Tasks performed to resolve the case, according to guideline15% 
Physical examination (PE)  3–4 questions  15% 
Complementary diagnostic means (Tests)  4–5 questions  30% 
Therapy (Tx)  4–5 questions  30% 
Diagnosis (Dx)  1–2 questions  1 question  10% 

To avoid selection biases, in each assessment session, participants were randomized into 2 groups, according to a cross-study design model. One group started being evaluated by MCQ and finished with VIP, the other vice-versa. At the end, participants were asked to fill out a satisfaction questionnaire.

Multiple-choice questions (MCQ)

The MCQ tests were solved online and built according to the recommendations.3,5,18,19 The MCQ followed the model of single best answer among 5 possible options. They were scheduled to be performed in about 20min, with the following order of linked questions: Clinical History (Hx), Physical Examination (PE), Complementary Diagnostic Means (Tests), Therapy (Tx) and Diagnosis (Dx). Participants were not allowed to go back to the previous question.

Virtual interactive patient (VIP)

We use BodyInteract™ as a VIP method. This is a medical simulator created in Portugal capable of creating an immersive experience, in which the user assumes the role of physician. VIP cases were scheduled to be resolved in 20min. Simulation ends when all recommended tasks are performed according to guidelines, the patient dies, or when this time ends. After solving the case, the participants had access to feedback on their performance.

Satisfaction questionnaire

A satisfaction survey was used, with the consent of its author.1 The questionnaire aimed to analyze the students’ experience and opinion about their performance in each assessment method.

Statistical analysis

The statistical analysis was performed using the SPSS Statistics 27 and the GraphPad Prism 9.

Descriptive statistical techniques were used: mean and standard deviation for the final scores obtained in each method (VIP and MCQ) and scores by parameters (Hx, PE, Tests, Tx and Dx) of each method of all participants.

We use the paired t-test for the null hypothesis between the difference in the final score between the methods and the difference between the same parameters of each method.

In the analysis of the satisfaction questionnaire, given that each participant could have answered the questionnaire several times, depending on the number of evaluation sessions they participated, the mean and standard deviation of points attributed to each item of the questionnaire per participant and then a paired t-test was performed for the null hypothesis between the different points given in each item of each evaluation method.

ResultsVIP and MCQ methods

The mean final score of the participants was higher with the VIP compared to the MCQ (VIP: 73.34±10.82 vs. MCQ: 67.40±8.96; p<0.01). Statistically significant differences were found in the Clinical History (VIP: 80.88±24.42 vs. MCQ: 69.70±19.30; p<0.5) and in the Physical Examination (VIP: 87.92±1.09 vs. MCQ: 64.03±19.21; p<0.001), where this was superior in the VIP. No statistically significant differences were found for the remaining parameters (Fig. 1).

Figure 1.

Mean score obtained by the participants, by parameter and grouped by methodology.

(0,1MB).
Satisfaction questionnaire

Participants consider that both assessment methods are adequate and satisfactory. VIP allows for an improvement in knowledge, as well as a better approximation to the clinical reality (Table 2).

Table 2.

Satisfaction questionnaire: the results are expressed as mean and standard deviation of an ordinal numeric variable between 1 (strongly disagree) to 7 (strongly agree).

Question  MCQ  VIP    p-value 
Did you have difficulties to answer the questions (MCQ)/to solve the clinical case (VIP)?  5.59 (±0.64)  5.21 (±0.98)    <0.05 
Were you able to concentrate to answer the questions (MCQ)/solve the clinical case (VIP)?  5.93 (±0.83)  5.64 (±1.06)    0.11 
Was solving the test (MCQ)/clinical case (VIP) stressful?  3.19 (±1.58)  5.18 (±1.52)    <0.001 
Do you feel satisfied with your performance in the test (MCQ)/clinical case (VIP)?  4.83 (±1.07)  4.64 (±1.37)    0.47 
Do you think that this evaluation method represents an adequate evaluation?  5.39 (±0.91)  5.39 (±1.23)    0.98 
Do you consider that your knowledge improved after the test (MCQ)/clinical case (VIP)?  5.17 (±1.30)  5.96 (±0.90)    0.001 
Are you satisfied with this evaluation method?  5.54 (±0.84)  5.78 (±0.93)    0.15 
Do you consider that the test (MCQ)/Clinical Case (VIP) is close to the clinical reality?  5.46 (±1.09)  6.24 (±0.61)    <0.001 

*Represents p-value <0.5 and *** represents p-value <0.01.

Discussion

It is expected that the use of VIP will play an important role in medical education in the coming years, both in the learning and evaluation processes, both for medical students and other health professionals.15 The use of equal parameters between both methods allowed the comparison capability in this study.

Our investigation suggests that medical students have better final scores when assessed by the VIP than by the MCQ. However, our study is limited by the fact that it did not take into account the tasks performed incorrectly when approaching the clinical case by the VIP, which could have harmful effects in real life.

Although our study has limitations in terms of the sample of participants that is not statistically significant and students participate voluntarily, these facts can compromise the quality of our study data. VIP has been well accepted by medical students, as has already been shown.16

In this study it was also possible to confirm 2 basic pillars of simulation in medicine with the VIP, approximation of the clinical reality and improve knowledge after the assessment, by feedback.20

Our investigation is favorable to the use of the VIP for the evaluation process of medical students in clinical years. A systematic review points out that the VIP proves to be more effective in improving clinical skills and knowledge.8 However, we know that the methods used in the assessment process in medical education have an impact on learning and have unintended effects, limitations and cultural trends.21 Therefore, a longitudinal assessment process combined with various methods must be implemented, in order to allow teachers to identify and correct the learning needs of medical students for the training of educated physicians, efficiently and with a good relationship cost-effectiveness.6,21 VIP is already used and shared among medical schools in the United States of America and Canada.22

In the future, it is suggested that an equitable standard system for scoring the performance of students when assessed using the VIP remains to be validated,15 as well as a fair penalty system for performing tasks that are incorrect or less cost-effective should also be considered.

Currently, the COVID-19 pandemic is affecting medical education, that must be seen as opportunities for change for medical education, based on teaching knowledge and ignore clinical reasoning and professional skills.23 The suspension of medical education has harmful consequences for the future of health care. To ensure the safety/health of medical students during the pandemic, it is necessary to ensure the continuity of their educational process and evaluation.

Quickly, all medical education has been adapted into online formats.24 Thus, the need to implement new innovative processes in medical education may require deep changes in many disciplines of the medical curriculum, thus contributing to a scenario of innovation and transformation in the medical curriculum. With new virtual educational systems will certainly play an important role as a support tool for the continuity of the educational and evaluation process in worldwide.24,25

Funding

This article was not funded.

Others information's

Article was made to obtain the Master's Degree in Medicine by the corresponding author from the Faculty of Health Science from University of Beira Interior (FCS-UBI), Covilhã, Portugal.

Abstract was accepted at the 7th Portuguese Nacional Congress os Pedagogical Practices in Higher Education (CNaPPES.21) for an oral communication of the work developed.

Conflict of interest

The authors of this article declare no conflict of interest.

Acknowledgements

Elisa Cairrão, Assistant Professor at the Faculty of Health Science from University of the Beira Interior (FCS-UBI) and Investigator at the Health Sciences Research Centre of the University of Beira Interior (CICS-UBI), Covilhã, Portugal.

Regina Rodrigues, Anesthesiologist at the Hospital Central do Funchal and Director of the Madeira Clinical Simulation Center, Madeira Island, Portugal.

References
[1]
J. Adjedj, G. Ducrocq, C. Bouleti, L. Reinhart, E. Fabbro, Y. Elbez, et al.
Medical student evaluation with a serious game compared to multiple choice questions assessment.
JMIR Ser Games, 5 (2017), pp. e11
[2]
U.M. Waldmann, M.S. Gulich, H.P. Zeitler.
Virtual patients for assessing medical students – important aspects when considering the introduction of a new assessment format.
Med Teach, 30 (2008), pp. 17-24
[3]
Z. Amin, C.Y. Seng, K.H. Eng.
Practical guide to medical student assessment [Internet].
World Scientific, (2006),
[4]
T. Ryall, B.K. Judd, C.J. Gordon.
Simulation-based assessments in health professional education: a systematic review.
J Multidiscip Healthc [Internet], 9 (2016), pp. 69-82
[5]
P.A. Coughlin, C.R. Featherstone.
How to write a high quality multiple choice question (MCQ): a guide for clinicians.
Eur J Vasc Endovasc Surg, 54 (2017), pp. 654-658
[6]
R.L. Yang, D.A. Hashimoto, J.D. Predina, N.M. Bowens, E.M. Sonnenberg, E.C. Cleveland, et al.
The virtual-patient pilot: testing a new tool for undergraduate surgical education and assessment.
J Surg Educ, 70 (2013), pp. 394-401
[7]
T. Poulton, C. Balasubramaniam.
Virtual patients: a year of change.
Med Teach, 33 (2011), pp. 933-937
[8]
A.A. Kononowicz, L.A. Woodham, S. Edelbring, N. Stathakarou, D. Davies, N. Saxena, et al.
Virtual patient simulations in health professions education: systematic review and meta-analysis by the Digital Health Education Collaboration.
J Med Internet Res, 21 (2019), pp. e14676
[9]
M. Botezatu, H. Hult, M.K. Tessma, U.G.H. Fors.
Virtual patient simulation for learning and assessment: superior results in comparison with regular course exams.
Med Teach, 32 (2010), pp. 845-850
[10]
F. Consorti, R. Mancuso, M. Nocioni, A. Piccolo.
Efficacy of virtual patients in medical education: a meta-analysis of randomized studies.
Comput Educ, 59 (2012), pp. 1001-1008
[11]
C.F. Wahlgren, S. Edelbring, U. Fors, H. Hindbeck, M. Ståhle.
Evaluation of an interactive case simulation system in dermatology and venereology for medical students.
BMC Med Educ, 6 (2006), pp. 40
[12]
N. Saleh.
The value of virtual patients in medical education.
Ann Behav Sci Med Educ, 16 (2010), pp. 29-31
[13]
J. Cendan, B. Lok.
The use of virtual patients in medical school curricula.
Adv Physiol Educ, 36 (2012), pp. 48-53
[14]
S. Huwendiek, B.A.D. Leng.
Virtual patient design and curricular integration evaluation toolkit.
[15]
D.A. Cook, M.M. Triola.
Virtual patients: a critical literature review and proposed next steps.
[16]
N. Gesundheit, P. Brutlag, P. Youngblood, W.T. Gunning, N. Zary, U. Fors.
The use of virtual patients to assess the clinical skills and reasoning of medical students: initial insights on student acceptance.
Med Teach, 31 (2009), pp. 739-742
[17]
M. Botezatu, H. Hult, U.G. Fors.
Virtual patient simulation: what do students make of it? A focus group study.
BMC Med Educ, 10 (2010), pp. 91
[18]
D.E. Campbell.
How to write good multiple-choice questions.
J Paediatr Child Health, 47 (2011), pp. 322-325
[19]
D. Pugh, A.D. Champlain, M. Gierl, H. Lai, C. Touchie.
Using cognitive models to develop quality multiple-choice questions.
Med Teach, 38 (2016), pp. 838-843
[20]
J.A. Windsor.
Role of simulation in surgical education and training.
ANZ J Surg, 79 (2009), pp. 127-132
[21]
R.M. Epstein.
Assessment in medical education.
[22]
G. Huang, R. Reynolds, C. Candler.
Virtual patient simulation at U.S. and Canadian medical schools.
Acad Med, 82 (2007), pp. 446-451
[23]
J.C. Ribeiro, T. Villanueva, A. Gi, P. Escada.
Constraints lead to opportunities for medical education in times of COVID-19 pandemic.
Acta Med Port, 33 (2020),
[24]
S. Rose.
Medical student education in the time of COVID-19.
J Am Med Assoc, 323 (2020), pp. 2131-2132
[25]
S. Tabatabai.
COVID-19 impact and virtual medical education.
J Adv Med Educ Prof, 8 (2020), pp. 140-143
Copyright © 2021. Elsevier España, S.L.U.. All rights reserved
Opciones de artículo
Herramientas
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos