Perceptions of students, professors, and simulated patients with a periodontology OSCE station in dentistry

Abstract

Background

The OSCE, or objective structured clinical examination, is a practical assessment used worldwide to evaluate students' competencies in health science degrees. It consists of stations, where different clinical situations can be replicated. Sometimes, the settings can introduce actors who perform the role of standardised patients. This study is aimed to determine the perception of students, teachers, and actresses involved in a periodontology OSCE station with a standardised patient.

Methods

In April 2023, 112 fifth-year dentistry students, 6 teachers, and 21 actresses participated in the study. The teachers responsible for the station design created three online questionnaires, distributed to all participants. The questionnaires consisted of a perception survey about the periodontology OSCE station in order to find out teachers, students and actresses` opinions about the station. The questions were divided by general demographics data (age, gender, and qualifications), 5 specific questions about the periodontology station and one open-ended question to express any consideration not mentioned previously.

Results

The results showed that most students handled the patient situation well (58.03%), found it realistic (83.92%), and appreciated the presence of a standardised patient (57.14%). Similarly, the teachers were satisfied (66.66%) or very satisfied (33.33%) with the exam, while the standardised patients generally felt satisfied (71.42%) or very satisfied (19.04%) with their role. Some patients experienced difficulties with evaluating and interpreting the checklist (57.14%).

Discussion

Overall, the organisation, explanations, and information provided for the stations were deemed accurate and satisfactory.

Resumen

Introducción

El ECOE, o examen clínico objetivo estructurado, es una evaluación práctica utilizada a nivel mundial para evaluar las competencias de los estudiantes de las carreras de Ciencias de la Salud. Consta de estaciones, donde se pueden recrear diferentes situaciones clínicas. En ocasiones, los escenarios pueden introducir actores que desempeñan el papel de pacientes simulados. Este estudio tiene como objetivo determinar la percepción de estudiantes, profesores y actrices involucrados en una estación de periodoncia en un ECOE, con un paciente simulado.

Material y métodos

En abril de 2023, participaron en el estudio 112 estudiantes de quinto año de odontología, 6 profesores y 21 actrices. Los profesores responsables del diseño de la estación crearon tres cuestionarios en línea, distribuidos a todos los participantes. Los cuestionarios consistieron en una encuesta de percepción sobre la estación de periodoncia del ECOE, con el fin de conocer las opiniones de profesores, estudiantes y actrices. Las preguntas se dividieron en datos demográficos generales (edad, género y calificaciones), 5 preguntas específicas sobre la estación de periodoncia y una pregunta abierta para expresar cualquier consideración no mencionada anteriormente.

Resultados

Los resultados mostraron que la mayoría de los estudiantes manejaron bien la situación del paciente (58,03%), la encontraron realista (83,92%) y apreciaron la presencia de un paciente simulado (57,14%). De manera similar, los profesores se mostraron satisfechos (66,66%) o muy satisfechos (33,33%) con el examen, mientras que los pacientes simulados en general se sintieron satisfechos (71,42%) o muy satisfechos (19,04%) con su rol. Algunos pacientes experimentaron dificultades para evaluar e interpretar la lista de verificación (57,14%).

Discusión

En general, la organización, las explicaciones y la información proporcionada para las estaciones se consideraron precisas y satisfactorias.

Background

The OSCE or objective structure clinical examination is a practical test that is universally accepted to assess the students’ competencies in health care degrees.^1,2

In the late 90s, the OSCE was used to assess clinical skills for the first time in dental school.³ It simulates a real situation in a standardised way, thus avoiding the possible examiner subjectivity evaluation.^4,5

The OSCE in health degrees are made up of stations.⁶ These are defined as simulated clinical scenarios with specific objectives in which the skills of the students are evaluated in different competencies such as the diagnosis, clinical judgement, or patient's management in a standardised way.⁷ In addition, it has been established that OSCE are useful to prepare the students to handle both: normal and rare situations which are not usual in real life.⁸

Sometimes, there are actors who take part in OSCE stations. The standardised patient has a checklist made by the station's designers, through which they can provide personal judgements about professional skills such as empathy and oral communication. The goal of the list is to achieve standardisation, always taking that into consideration and prioritising it above other factors of influence.⁵

Apart from the checklist, the actors are given scripts that are intended to replicate, not only a complete medical history, indicating medical records as - diseases or medication-, but a specific behaviour previously agreed with the station's designers. This will allow to evaluate different student's skills beyond their dentistry theoretical knowledge. All that will contribute to a more complete student evaluation, analysing the different competencies beyond the theoretical knowledge gained during the degree's program.⁵

The OSCE is currently considered the gold-standard of capacities’ test, providing students the comfort of facing a station designed in a safe and controlled way, as similar as possible to a real case in a dental practice.⁷

The purpose of this work was to determine the students, professors and standardised patients’ satisfaction level resulting from a periodontology station in the OSCE at our university.

Methods

The approval was granted by the University Ethics Board and was assigned the internal code CIPI 22.238.

The periodontology OSCE station was designed in 2022 by the Department of Clinical Dentistry's teaching staff. The study methodology and design were previously published.⁹ The OSCE consisted of 4 additional stations, apart from the periodontics station, and each station had to be completed in 8 min. Due to the large number of students who had to take the exam, each periodontics station was carried out simultaneously in 4 identical scenarios. In addition, the OSCE was held over 2 consecutive days. Therefore, the university recruited 21 actresses, who were exchanged throughout the OSCE.

Students are familiar with the OSCE assessment simulation. They also carry out various simulations in other subjects throughout the bachelor's degree.

The pathology selected by the station's designers was necrotizing gingivitis (NG). The reason for choosing this disorder was that, due to its low incidence of occurrence, students would not have seen it during their clinical practices with patients.

Considering that a pilot study had previously been conducted,⁹ the designers included some modifications in order to improve the reliability during the evaluation.

Before the simulation took place, the station's designers had a meeting with the actresses to standardise the test, verify the checklist, emphasise the behaviour outlined on the script that they should have in front of the students and, finally, answering their questions. They were also instructed on how to carry out the assessments of the students on the Uevalua platform. These assessments are carried out when each student had left the station. The standardised patients had 2 more minutes time for complete it.

In this academic year, it was decided that the standardised patients' briefing was given by the teachers who designed the station, being one of the adjustments introduced with respect to the 2022 OSCE. In previous years, the person in charge of the actors’ briefing was the OSCE coordinator instead of the station designer.

Each student, prior to the interview with the standardised patient, was provided with the patient's relevant medical and dental history information, signs and symptoms, as well as the cause of the dental consultation. The description of the intraoral examination was also detailed, and they were informed that an intraoral photograph and an X-ray of the patient were available.

This time, the students came into the room where the actress was. On the table, there was a coloured photography where they could watch characteristic NG teeth and gums; also, a periapical radiography of the patient's upper central incisors. The student was not required to perform an intraoral examination of the patient, but to provide the presumptive diagnosis and propose a complete treatment plan.

At the end of the OSCE, the standardised patients must grade the student based on the skills competencies checklist: communication abilities, diagnosis judgement, therapeutic judgement, prevention, and ethical-legal considerations. A score was assigned to each of these items according to their degree of importance in the station, with the highest being a 3 and the lowest a 1, obtaining a total of 47 points in the end, with at least 50% of the total score to pass (Table 1).

Table 1.

Items included on the evaluation periodontology station.

Item	Questioned/explored/performed	Detailed description	Yes	No	Score
1	Empathy, assertiveness	Greets me politely (shakes my hand, address me using “you”)			3
2	Assertiveness	Listens properly, does not interrupt the patient, looks at me while talking			2
3	Respect	Does not make criticisms or pejorative comments to the patient			3
4	Respect and interest	Is interested in the opinions, concerns, or emotions of the patient's parent			2
5	Diagnosis	The student informs me that the pathology is necrotizing gingivitis (NG)			3
6	Medical history. Stress and anxiety	Tells me that stress influences the development of this pathology			3
7	Medical history. Stress and anxiety	Discuss that stress causes immunosuppression and therefore favours the appearance of this pathology			1
8	Medical history. Epidemiology	Discloses that sex does not influence the development of pathology			1
9	Medical history. Stress and anxiety	The student comments that anxiolytic medication does not affect the development of the disease			3
10	Symptoms and diagnostic signs	Explains clearly what the pain and bad breath are due to			3
11	Symptoms and diagnostic signs	Informs me clearly why the papillae are not right			2
12	Treatment	Explains me that scale (prophylaxis) is the treatment of choice			3
13	Treatment	Postpone a week the prophylaxis due to severe pain			3
14	Treatment	Prescribes me rinses with Chlorhexidine 0.12% every 12 h until the next appointment			3
15	Antibiotic therapy	I am prescribed Metronidazole 250 mg × 2 every 8 h			3
16	Differential diagnosis	Knows and explains the difference between gingivitis and necrotizing gingivitis			3
17	Differential diagnosis	Explains what it is necrotizing periodontitis and why this is not			3
18	Informed consent	Give the patient the IC of periodontics treatment			3

The teachers in the study were divided into two groups: the staff who designed the station and the study, and the professors who participated as observers in the OSCE and completed the perception surveys. At the same time as the OSCE was taking place, professors were observing and listening to the test from a control room adjacent to the station. In this way, they were able to evaluate the students' reactions. Neither standardised patients nor students were aware of the hidden presence of the teachers.

This test was taken by 220 students of the 5th year of Dentistry in April 2023; also 6 teachers and 21 actresses participated in it.

The teachers involved in the periodontology station developed 3 online questionnaires, which were given in different moments to all the participants, both students, actresses and teachers (Table 2). The questionnaires were conducted using Microsoft Office Forms.

Table 2.

Items corresponding to the three questionnaires with response options.

Students' questionnaire

QUESTION 1. There was a standardised patient in the periodontology station, do you think that having completed two years of internship with patients at the university dental practice could help you manage the patient's anxiety? – I attended to the patient without problem – Although she was a difficult patient, I was able to assist her. – It was difficult for me to manage this sort of patient – I was not able to handle the patient

QUESTION 2. The periodontology station was carried on with a standardised patient, do you think this kind of pretended scenario could happen in the real practice? – The situation shown at the station could certainly take place in the real dental practice – There are no patients with such a difficult attitude in the real dental practice – The situation arranged at the OSCE was not real – There is no such complex periodontal pathology in the real dental practice

QUESTION 3. What of the possible methodology to be used in the OSCE do you think would be closer to the real dental practice and would make you better reflect your professional skills? – Clinical case – Standardised patient – Clinical images of lesions – Radiographies interpretation

QUESTION 4. What do you consider most complicated of the periodontology station with the standardised patient? – Anxiety management – Treatment plan – Correct diagnosis – Planning periodontal check-ups

Teachers' questionnaire

QUESTION 1. After your participation in the periodontology station, what do you think is the most appropriate assessment method in terms of clinical skills and patients' management for the fifth-year students at the university? – Written exam with short question or multiple choice – Oral exam – Clinical case – OSCE

QUESTION 2. Do you think the periodontology station could accurately simulate a real situation at the dental practice? – The situation presented is true to reality – It could be more customised to the dental practice reality – It would never accurately reflect the everyday reality of the dental practice – Do not know/Do not answer

QUESTION 3. As a teacher, which is your satisfaction level with the OSCE test? – Very satisfied – Satisfied – Not very satisfied – Unsatisfied

QUESTION 4. As a teacher, what is your perception of the students’ satisfaction? – It is an assessment test, and the students are very satisfied with this type of exam – The students have the same attitude as they do to any other assessment test – I do not think they are satisfied with this sort of tests – I cannot evaluate the students’ satisfaction

Standardised patients' questionnaire

QUESTION 1. As an actress, how satisfied are you overall with the OSCE? – Very dissatisfied – Not very satisfied – Satisfied – Very satisfied

QUESTION 2. Do you think this station would show a common situation in the dental practice? – I do not think this situation would ever happen in the dental practice – I think it is possible that this situation occurs in the dental practice – I think it is very usual for this situation to happen in the dental practice – Do not know/Do not answer

QUESTION 3. As an actress, what has been the most complicated part of the periodontology OSCE's training? – Scientific terms – Understand the average score table – Handle the students’ question – Others

QUESTION 4. As an actress, what is your opinion regarding the evaluation average score table of the periodontology OSCE? – The average score table is very clear, and I have had no problem assessing the students – The average score table leaves room for doubts, and it was no easy to fill it – The average score table is complex to fill out by someone who is not an expert in this area – Other

Questionnaire of the students

Two weeks after the OSCE, the students filled out a periodontology station perception survey.

In this case, the students’ perception survey could not be completed immediately after the OSCE due to the classroom shape, students' number, and station-specific characteristics.

The questions were divided into general demographics data and 5 specific questions about the periodontology station with four choices. Finally, an open question was included.

Questionnaire of the teachers

The same day of the test, the teachers filled out a survey specially designed for them.

In this case, despite they were present at the exam, none of the station's designers answered the questionnaire, or collected the professors’ surveys to avoid data bias.

The questions were divided into general demographics data and 4 specific questions about the OSCE station with four choices. Lastly, an open question was included.

Questionnaire of the standardised patients

The survey was completed by the actresses who had participated on the OSCE periodontology station on the same day the test was taken.

Three questions about the OSCE station were included with four choices. Finally, an open question was included.

UEVALUA platform

The skill competencies checklist was integrated into the UEVALUA digital platform. Once graded, students can access their scores through this platform, allowing them to identify areas of failure and understand how to improve for the future. This tool provides feedback that supports their academic development.

Statistical analysis

The answers were exported to a Microsoft Excel sheet, which was the software used for their analysis. A descriptive statistic of the participants' sample was made. The absolute and relative frequencies referring to each answer option in each of the questions were calculated.

Results

Results from students’ questionnaire

A total of 220 students conducted the OSCE. The questionnaire was answered by 112 students, so the response rate was 50.90%. Of the study's participants, 50% were men and the other 50% were women and their mean age was 25.87 ± 5.39 years old.

Table 3 shows the results obtained in the students’ questionnaire.

Table 3.

Students' questionnaire results.

	n	%
QUESTION 1. There was a standardised patient in the periodontology station, do you think that having completed two years of internship with patients at the university dental practice could help you manage the patient's anxiety?
I attended to the patient without problem	65	58%
Although she was a difficult patient, I was able to assist her.	37	33%
It was difficult for me to manage this sort of patient	10	8.9%
I was not able to handle the patient	0	0
QUESTION 2. The periodontology station was carried on with a standardised patient, do you think this kind of pretended scenario could happen in the real practice?
The situation shown at the station could certainly take place in the real dental practice	94	83.9%
There are no patients with such a difficult attitude in the real dental practice	5	4.5%
The situation arranged at the OSCE was not real	6	5.4%
There is no such complex periodontal pathology in the real dental practice	7	6.3%
QUESTION 3 What of the possible methodology to be used in the OSCE do you think would be closer to the real dental practice and would make you better reflect your professional skills?
Clinical case	24	21.4%
Standardised patient	64	57.1%
Clinical images of lesions	16	14.3%
Radiographies interpretation	8	7.1%
QUESTION 4. What do you consider most complicated of the periodontology station with the standardised patient?
Anxiety management	28	25%
Treatment plan	23	20.5%
Correct diagnosis	50	44.6%
Planning periodontal check-ups	11	9.8%

In regard to the open question where the students could give their opinion about the station with standardised patients, the feedback was positive overall, with answers such as “very useful”, “strength to have actors”, “I am happy”. Suggestions for the next academic year were given on the survey, such as “it was not very useful if compared with real patients”, “anxiety”, or “very nervous due to lack of time”.

Results from teachers’ questionnaire

The periodontology OSCE station was supervised by 6 teachers and all of them completed the satisfaction survey. Their answers are shown in Table 4.

Table 4.

Teachers' questionnaire results.

	n	%
QUESTION 1. After your participation in the periodontology station, what do you think is the most appropriate assessment method in terms of clinical skills and patients' management for the fifth-year students at the university?
Written exam with short question or multiple choice	0	0
Oral exam	0	0
Clinical case	1	16.7%
OSCE	5	83.3%
QUESTION 2. Do you think the periodontology station could accurately simulate a real situation at the dental practice?
The situation presented is true to reality	2	33.3%
It could be more customised to the dental practice reality	4	66.7%
It would never accurately reflect the everyday reality of the dental practice	0	0
Do not know/Do not answer	0	0
QUESTION 3. As a teacher, which is your satisfaction level with the OSCE test?
Very satisfied	2	33.3%
Satisfied	4	66.7%
Not very satisfied	0	0
Unsatisfied	0	0
QUESTION 4. As a teacher, what is your perception of the students’ satisfaction?
It is an assessment test, and the students are very satisfied with this type of exam	3	50%
The students have the same attitude as they do to any other assessment test	3	50%
I do not think they are satisfied with this sort of tests	0	0
I cannot evaluate the students’ satisfaction	0	0

Regarding the open question where the teachers could express their opinion about the station with standardised patient, all the answers were positive. Some examples are shown here below: “These tests help to prepare the students for the real-life”, “The assessment is very interesting; in particular to assess special clinical conditions that may not happen in the clinical practices at University”. It should be noted that some of the teachers made suggestions for future OSCE test: “The students should be evaluated by the teachers. In this way, I think the actors would be more focus on acting and would not guide students through the questions”.

Results from standardised patients' questionnaire

The 21 actresses who participated in the periodontology station filled in the perception questionnaire. Their opinions are shown in Table 5.

Table 5.

Standardised patients' questionnaire results.

	n	%
QUESTION 1. As an actress, how satisfied are you overall with the OSCE?
Very dissatisfied	0	0
Not very satisfied	2	9.5%
Satisfied	15	71.4%
Very satisfied	4	19%
QUESTION 2. Do you think this station would show a common situation in the dental practice?
I do not think this situation would ever happen in the dental practice	0	0
I think it is possible that this situation occurs in the dental practice	8	38.1%
I think it is very usual for this situation to happen in the dental practice	13	61.9%
Do not know/Do not answer	0	0
QUESTION 3. As an actress, what has been the most complicated part of the periodontology OSCE's training?
Scientific terms	6	28.6%
Understand the average score table	6	28.6%
Handle the students’ question	1	4.8%
Others	11	47.6%
QUESTION 4. As an actress, what is your opinion regarding the evaluation average score table of the periodontology OSCE?
The average score table is very clear, and I have had no problem assessing the students	5	23.8%
The average score table leaves room for doubts, and it was no easy to fill it	12	57.1%
The average score table is complex to fill out by someone who is not an expert in this area	2	9.5%
Other	2	9.5%

By having an open answer and getting a majority of “others” on the last question, they were asked to develop their answer. In general, most of the problems were about assessing the students and following the script at the same time. Some of their comments were: “Having to be aware of evaluating the students and interpreting the role”, “Grading students meanwhile acting is complicated”, or “Remembering all the data providing by the student”, “It would be easier to fill in the evaluation sheet if we were not acting simultaneously”, or “I think that 1 minute to fill the evaluation sheet is not enough in case there are some problems with the tablet. Stack up students”.

With respect to the open question where the actresses could express their opinion about the OSCE station, the responses were generally positive, with answers like “The periodontology OSCE has been enjoyable, with needed breaks every 5 students. “The teachers have been very nice”, “I have really liked it and I have felt very comfortable”. There were also suggestions obtained for next years: “The role is very easy to perform, but the difficulty of filling the evaluation form is added”, “I think it is a fantastic didactic idea. I suggest leaving more time between students so we can concentrate on the evaluation”, or “Having to do the evaluation (rating)”.

Discussion

Among the basic criteria for an effective assessment tool such as the OSCE are the validity, reliability, and feasibility of the tool. However, to guarantee the success of the evaluation process, other factors must be considered like the educational potential and acceptance by people who manage, implement, and assess the process.¹⁰

For these reasons, it was decided to evaluate the perception, not only of the students, as it had been done before, but also to know the impression of the staff involved in 2023 assessment. In this case: teachers and actresses.

On the other hand, OSCE provides an opportunity to apply the knowledge in scenarios very close to real life. Therefore, to guarantee the continuity of this type of assessments, it is essential to know whether this assumption is fulfilled through the estimation of the staff involved.

It is already known, from an educational point of view, that students greatly appreciate that the OSCE allows them to think, integrate their knowledge and preparate for the next step in their education.¹¹ Through exploring the different perspectives of the stakeholders, the study highlights the potential of this type of test that combines the expertise and contributions of diverse participants to achieve a successful implementation.¹²

The results show a general student's perception.

It is worth highlighting the impact of the presence of a standardised patient at the station. This in fact enhances the test, as it encourages the improvement among the students, and allows for optimal evaluation on the communication skills, the empathy, and the patients’ management. More than 50% of the surveyed students preferred the standardised patient station over any other assessment method.

These results are similar to those found in the literature. In 2014 at a university in Iran, the satisfaction level of students with the OSCE of internal medicine was measured and compared with other types of exams. They obtained very similar results to those shown by our students.¹³

In contrast to these results, a study conducted by other authors in 2018 did not obtain the expected outcomes, as many of the surveyed students did not believe in the reliability and standardisation of the test. They had the perception that the quality, validity, and reliability of it was low. In addition, they found it to be stressful and intimidating.¹⁴

Although stress and anxiety measurement were not part of the survey itself, some of the students reflected on the open-ended question to feel slightly stressed with the time provided or the actress's questions.

Despite the fact that our students had previous experience in other OSCEs, had participated in simulations in other courses and had practised with real patients for two years, 25% of them indicated that anxiety management was the most challenging aspect of the exam. Troncon also noted that medical students with high levels of anxiety before an OSCE, failed to reduce their anxiety with increased experience on this test, possibly due to fear of failing.¹⁵ In addition, the interactive and timed aspects of the OSCE can generate high levels of anxiety in students.¹⁶

There are multiple similarities between the results obtained by our students from and Porto et al. in 2023.¹⁰ In their case, a pilot study with 14 students was conducted, where both virtual and on-site OSCEs were carried out. Their survey was made immediately after the OSCE. Regarding the perceptions obtained by their students and the anxiety levels, no differences were shown between the two analysed models, with acceptable values in both cases and high satisfaction from the students.¹⁰

In the same line, regarding the overall students’ perception, although in another area, Ataro et al., in 2020, obtained a similar finding by asking closed-ended questions to their students, and then as in the present study, allowing participants to express their impressions through elaboration. Overall, the students were happy and excited about having the opportunity to opt for this evaluation system compared to the traditional ones. Students had reported higher levels of anxiety during OSCE station because of the presence and direct observation of an examiner.³ Kerkstra et al.³ aimed to determine whether a video-monitored OSCE would create less anxiety than direct observation. The results showed low to moderate anxiety with both methods, as well as lower-than-expected baseline levels of anxiety.

In this same line, in a review conducted by Chan et al. in 2022,¹⁸ they established that the students have less anxiety when taking a virtual OSCE test compared to taking the same type of test in a classroom. This type of investigation has experienced significant improvement due to the need of alternative assessment methods during the COVID-19 pandemic.^18,19

In 2020, Getu Ataro et al. conducted a study to implement OSCE in a medical school. This is one of the few studies in which the gratification of the teachers involved in the station was considered. In addition, the sample size is similar, since the surveys filled by 7 teachers were analysed. All the teachers were satisfied with both the type of exam and the way it was designed. In this case, they complained about organisational issues, which differs from our study where a general level of satisfaction was achieved in each of the items the involved staff was asked about.¹⁷

We have found the data obtained by Chahine et al. interesting.²⁰ In this research, the purpose was to evaluate the impressions and the cognitive processes associated with the examiners from a medical OSCE. Instead of focusing on their level of satisfaction, feedback in terms of test's standardisation and examinations` judgement and objectivity were obtained. However, it might be interesting to expand further research avenues in which the evaluations got from all staff involved in the test are assessed.

In terms of actress’ satisfaction, good outcomes were obtained. The collected data show that both in the organisation and the explanations, the information provided to develop the station was accurate and satisfactory.

When reviewing the literature, it has been found that the satisfaction of the standardised patients after their participation in this type of test has not been studied so far. This adds value to the results obtained in our surveys. However, it is necessary to increase the sample size in order to validate the results obtained.

One of the main characteristics of the present study, apart from this type of data, is the participation of a standardised patient, since previous OSCE tests have not always used them. Furthermore, the patient was the test's examiner. To accomplish this, the standardised patients had a checklist with the items that must be evaluated in a question-and-answer yes/no format.

The checklist was integrated into the UEVALUA digital platform. Once graded, the students have access to the score, and they can see each of the items that have been evaluated. Therefore, through UEVALUA, as in Manogue et al. study,²¹ feedback of qualifications between teachers and students is established; the students can see where they have failed and how they can improve for the future. The feedback concept was highly valued by the students, since the comments and items graded by the teachers could help them in their academic development.

Referring to the examiners, the literature shows us that it was the teachers, and not the standardised patients, who followed up with the students.²² The standardisation and checklist development is a distinctive OSCE characteristic as well as the subjectivity suppression of another sort of test.⁵ In addition, a sufficiently comprehensive prior briefing could guarantee that a person without knowledge on the topic, like the OSCE standardised patient, could or should have the capacity to evaluate in the same way as a teacher or other specifically trained person. Also opens up many possibilities in terms of fewer staff to perform these examinations and cost savings. The Mc Laughlin study demonstrates that standardised patients are reliable to assess OSCEs if they are well trained.²³

In some cases, highly protocolized calibrations were made. Yeates et al.²⁴ developed a station where the examiner's previous calibration was done. For that purpose, a student's exam video was shown so that each assessor could independently evaluate it. That allowed the evaluation criteria for each of the participating teachers to be standardised. The intention of this was to provide an objective and reliable evaluation. However, they considered that the evaluation could be inaccurate.²⁴

In subsequent studies, it should be determined whether this kind of tests work under the teacher's observation who independently evaluates the students, thus verifying the consistency of the assessments.

Limitations

The present study has some limitations which have manifested in the design and development of itself.

OSCE teachers' collaboration are reduced due to the facilities, the simulated hospital space (where the OSCE takes place) and the number of students each academic year. Because of these reasons, it is complicated to achieve an immediate questionnaire answer from the student, as well as the number of teachers involved in the OSCE. This can lead to several biases, such as blurring of recall, participation of only the most satisfied or most dissatisfied students, loss of sample, or the possibility of influence on their opinions due to the time available to discuss with their peers. For future studies, we will conduct the surveys immediately after the test, to avoid these distortions to a large extent.

Another aspect to be considered is the large number of standardised patients used to perform the OSCE, which may increase variability in the assessment and thus affect the reliability of the test. To improve consistency and reliability in future studies, uniform training and continuous supervision of standardised patients should be provided, and as far as possible with the same actors who already have experience in OSCEs.

The answers to the different questions are based on the previous year, where mistakes or misunderstandings were not found. This is the reason why it was replicated with a larger sample size and included both, teachers, and students.

As it has been abovementioned, the results of the pilot study were published, and after the literature review, it was observed that there were not enough publications regarding the teachers’ and actors’ opinion and satisfaction with this sort of test.

As faculty members, our commitment to the students and the constant desire for improvement led us to assess the present items in a qualitative way. The answer options to the questionnaires were made based on student's needs. It is understandable that they are a bit unclear, therefore according to the opinion and data collected, appropriate changes would be considered for future studies.

Informed consent

Informed consent was obtained from all subjects involved in the study.

Ethics approval

The study was approved by the Local Ethics Committee (protocol code CIPI 22.238).

Funding details

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.