Effects of attending workshops and using complementary material for home training on the learning of surgical knots by general surgery residents

Abstract

Introduction

In the medical field, specifically in surgical specialties, the knot is crucial for suturing tissues or organs, allowing wounds to heal. This study aims to evaluate the effect of having access to different learning tools for continuous practice on learning basic surgical skills.

Materials and methods

A surgical knots workshop was held, and a prospective randomized controlled trial (RTC) was performed. General surgery residents who assisted in the workshop and accepted to participate in the study were divided into two intervention groups (Groups A and B) groups through the block randomization technique. A control group (Group C) was formed with General surgery residents not assisting the workshop. Group A continued their residency activities, participated in the knot workshop, and received complementary teaching material for home training. Group B continued their residency activities and participated in the knot workshop. Group C simply continued their residency activities. The three groups were evaluated with the OSATS scale and a procedure-specific checklist before and after the knot workshop to assess and compare proficiency.

Results

Paired hypothesis testing showed that all groups significantly increased their skills over the study period. Unpaired hypothesis testing indicated that, although all groups showed improvements, Group A obtained statistically higher scores than Groups B and C on the post-training evaluations.

Conclusion

Access to complementary teaching material, in addition to residency activities and workshops, is an important support for surgeons' training.

Resumen

Introducción

En el ámbito médico, específicamente en las especialidades quirúrgicas, el nudo es crucial para suturar tejidos u órganos, permitiendo la cicatrización de las heridas. Este estudio tiene como objetivo evaluar el efecto de tener acceso a diferentes herramientas de aprendizaje para la práctica continua en el aprendizaje de habilidades quirúrgicas básicas.

Materiales y métodos

Se realizó un taller de nudos quirúrgicos y se realizó un ensayo controlado aleatorio (RTC) prospectivo. Los residentes de cirugía general que asistieron al taller y aceptaron participar en el estudio fueron divididos en dos grupos de intervención (Grupo A y B) mediante la técnica de aleatorización en bloques. Se formó un grupo control (grupo C) con residentes de Cirugía General que no asistieron al taller. El grupo A continuó con sus actividades de residencia, participó en el taller de nudos y recibió material didáctico complementario para la formación en casa. El grupo B continuó con sus actividades de residencia y participó en el taller de nudos. El grupo C simplemente continuó con sus actividades de residencia. Los tres grupos fueron evaluados con la escala OSATS y una lista de verificación específica del procedimiento antes y después del taller de nudos para evaluar y comparar su competencia.

Resultados

Las pruebas de hipótesis pareadas mostraron que todos los grupos aumentaron significativamente sus habilidades durante el período de estudio. Las pruebas de hipótesis no apareadas indicaron que, aunque todos los grupos mostraron mejoras, el grupo A obtuvo puntuaciones estadísticamente más altas que los grupos B y C en las evaluaciones posteriores al entrenamiento.

Conclusión

El acceso a material didáctico complementario, además de las actividades de residencia y talleres, es un apoyo importante para la formación de los cirujanos.

Introduction

The knot is “a tie that tightens and closes so that it can hardly be released on its own, and that the more you pull on any of the ends, the more it tightens”.¹ Suturing is the act of sewing or joining tissues through the use of a special thread in order to facilitate healing and prevent infection. In the medical field, specifically in surgical specialties, the knot is crucial for suturing tissues or organs, allowing wounds to heal.

The scientific advances that have existed in recent years have led us to seek progress in the way surgeons are taught.² The learning process of the resident surgeon begins with a series of knowledge previously acquired during undergraduate studies. The medical residency allows the doctor to consolidate and complement this knowledge with practical skills. This learning process is carried out through stages. In the initial phase, the individual observes an action, selects the most important aspects of it, and elaborates on how to replicate it through nervous impulses that register information in the brain. Observing the action again allows for improving previous information, correcting previous errors, and translating them into appropriate motor behavior. One action requires several repetitions until it is perfected through practice. Finally, the autonomous stage is reached in which the student knows how to perform a task without thinking about the procedure, allowing them to concentrate on other aspects or the next part of the procedure. Through constant practice, practical skills are created and honed. Likewise, by creating this dynamic, we move away from traditional teaching based on theoretical classes taught by a teacher, where only one-way learning is created.³

The evolution of medical education includes, in its advances, stricter supervision of work hours and a focus on proficiency and evaluation of skills.^4–7

Clinical simulation is a series of scenarios that replicate real situations where the theoretical knowledge previously seen is integrated into a practical situation. It is an innovation that allows the student to repeatedly practice the skills and abilities they need in a safe environment until they feel confident enough to perform on an actual patient. It is an effective strategy for resident surgeons who persist in continuous training as it increases the learning quality and allows the integration of knowledge with its consequent evaluation.^8,9

Several proposals have been developed for the evaluation of surgical skills. In the late 1990s, Martin et al.¹⁰ developed a scale called “Objective Structured Assessment of Technical Skills” (OSATS) at the University of Toronto. The OSATS is based on the direct observation of residents or surgeons performing different surgical procedures in simulation models. Currently, OSATS is used as an evaluation method where expert and certified analysts review the surgery, providing a final score of the operator's technical performance. Since its validation, multiple studies have used the OSATS scale to evaluate participant performance.^11–13 A complementary procedure-specific checklist is commonly used when evaluating surgical skills with the OSATS scale.^10,13

The objective of this study is to evaluate the effects of attending workshops and having access to simulators for home training in basic knotting skills acquisition by residents of surgical specialties using the OSATS scale and a procedure-specific checklist to measure proficiency before and after a training period.

Materials and methods

Population and location of the study

A surgical knot workshop was held at the author's institution. Two hundred thirty-two resident doctors from the first year of General Surgery with basic knowledge of manual surgical knots participated in the workshop.

All residents attending the workshop were invited to participate in the study. Residents were explained that if they accepted enrolling in the study, they would randomly be allocated into one of two groups: one that would be allowed to borrow the low-cost simulation model used at the workshop and one that would not. A total of 40 residents agreed to participate (17.24 % enrollment rate) and were divided into two groups of 20 participants each, Group A and Group B.

The study participants had the following characteristics: First-year residents of the General Surgery specialty comprised 42 men and 18 women aged 24–30 years. Residents were informed that if they participated in any other external simulation workshops during the study, they would be eliminated from the analysis to avoid bias. The study was approved by the ethics committee of the author's institution. All participants signed a letter of informed consent, and the study was carried out following the principles of the Declaration of Helsinki.

Procedure, study design, and groups

Local hospitals with medical residency programs in general surgery were contacted to invite them to participate in the knot workshop. In total, 29 hospitals agreed to participate.

Residents attending the workshop who agreed to participate in the study were randomly assigned to either Groups A or B using the block randomization technique, with ten blocks of four participants each.

Group A carried out the practical workshop under the teaching and supervision of expert tutors, using a low-cost simulation model they could take home after the workshop to practice different knots for 15 days. They also had access to a virtual platform with an electronic manual and videos about each knot elaborated by experts from the author's institution.

Group B attended the practical workshop, where they used the same low-cost model as Group A. However, they were not allowed to take the model home and did not have access to the electronic manual and videos.

A control group (Group C) was formed by 20 residents who did not attend the knot workshop, and neither had access to any additional tools or guidance beyond their normal residency activities.

All groups performed two evaluations: a pre-training evaluation before starting the surgical knots workshop and a post-training measurement two weeks after the workshop. The proficiency of each group was evaluated according to the OSATS scale (Table 1) and a proposed procedure-specific checklist specific to the surgeon's knot task (Table 2) on the simulator model. The pre-training and post-training evaluations were analyzed to search for differences between the proficiency achieved by the three groups.

Table 1.

OSATS scale.

OSATS: Global performance score scale
The global score used in the OSATS scale is a tool to grade the skills of surgeons. Maximum score for the 7-item scale is 35 points, and minimum score is 7.
Please mark with an X the score of each item according to the candidate’s performance, independently of the capacitation level
item	1	2	3	4	5
Respect for tissue	Frequently used unnecessary force on tissue or caused damage by inappropriate use of instruments		Careful handling of tissue but occasionally caused inadvertent damage.		Consistently handled tissues appropriately with minimal damage.
Time and motion	Many unnecessary moves		Efficient time/motion but some unnecessary moves		Economy of movement and maximum efficiency.
Instrument handling	Repeatedly makes tentative or awkward moves with instruments		Competent use of instruments although occasionally appeared stiff or awkward		Fluid moves with instruments and no awkwardness
Knowledge of instruments	Frequently asked for the wrong instrument or used an inappropriate instrument		Knew the names of most instruments and used appropriate instruments for the task.		Obviously familiar with the instruments required and their names.
Use of assistants	Consistently placed assistants poorly or failed to use assistants		Good use of assistants most of the time		Strategically used assistant to the best advantages all times.
Flow of operation and forward planning	Frequently stopped operating or needed to discuss next move		Demonstrated ability for forward planning with steady progression of operative procedure		Obviously planned course of operation with effortless flow from one move to the next.
Knowledge of specific procedure	Deficient knowledge. Needed specific instructions at most operative steps.		Knew all important aspects of the operation.		Demonstrated familiarity with all aspects of the operation.

Table 2.

Procedure-specific checklist.

Procedure-specific checklist. Surgeons knot on the low-cost simulation board.
Add a x mark at the appropriate column (YES / NO)	YES (1 point)	NO (0 points)
Adequately use of the rope
Doesn't leave work field
Minimize movements
Adequately uses all fingers of both hands to perform the knot
Doesn't generate traction of the model
Can perform the knot without guidance
Time taken to perform the knot is appropriate
Correctly positions the first loop
Correctly positions the second loop
The final knot is properly performed
Applied force is appropriate
Doesn't drop the loop at any time

Due to the non-randomized allocation of Group C, this study design falls in the classification of quasi-experimental study. The described methodology design for this study corresponds to the C1 level (Quasi-experimental study with untreated control group with pre-test and post-test samples) on the relative hierarchy of quasi-experimental study designs as reported by Harris et al.¹⁴

Before beginning the knot workshop, all participants enrolled in the study performed the surgeon's knot as part of their pre-training evaluation. After performing the pre-training evaluation of all participants, the knot workshop started. Residents attending the workshop received mentorship from expert surgeons of multiple institutions and had the opportunity to practice their skills under supervised guidance with the low-cost simulation model.

The theoretical-practical workshop was taught by expert general surgeons, who explained and demonstrated step by step the four most used manual knots in surgical practice. The one-handed square knot, the two-handed square knot, the surgeon's knot, and the deep knot were demonstrated and explained in detail. Teaching was supported with audiovisual materials on the online platform. All knots were made in a low-cost board simulator consisting of a board with a rope to tie knots (Fig. 1). To verify that the participants in Group A used the material at home, the virtual platform with the workshop material recorded user access.

Figure 1.

Low-cost simulation model used during the knot workshop.

After the demonstration by the experts, the resident surgeons practiced on the simulator under the tutors' supervision. This supervised training allowed the correction of errors and the resolution of doubts, with the aim that the participants could tie the knots correctly and independently.

The workshop lasted two hours. After the workshop, participants of Group A and residents not participating in the study were allowed to borrow the simulation model, while participants of Group B were not. After 15 days, participants enrolled in the study returned to the simulation model and performed their post-training evaluation, in which they repeated the surgeon knot.

Although Group C did not attend the workshop, it performed a pre-training evaluation followed by a post-training evaluation 15 days later, as Groups A and B did.

This study design allowed us to measure the impact of the surgical knots workshop and access to additional resources on residents' skill development.

Measured variables

The variables measured included in this study were the total scores from the OSATS scale and the procedure-specific checklist. Scores from these scales were used to evaluate the three groups’ pre- and post-training proficiency. The OSATS scale has been previously validated as an evaluation tool for multiple procedures including basic suturing skills^{10–13,15–17} so there was no need to revalidate it for this study.

Statistical analysis

Statistical analysis was performed using the statistical software R version 3.6 with the RStudio IDE. All collected sample sets were inspected with the Shapiro-Wilks test to confirm normality in data distribution. Student's t-tests (for normal populations) and paired Wilcoxon-Mann–Whitney test (for non-normal populations) were used to analyze differences between each group's pre-training and post-training proficiency. An analysis of variance (for normal populations) or Kruskall-Wallis (for non-normal populations) test was performed to identify differences between the groups' performance in their pre-training and post-training stages.

Results

All 60 residents who participated in the study completed the pre- and post-training evaluation. Table 3 presents the OSATS, and procedure-specific checklist scores in the pre-training and post-training stages of the three groups and the p-values of the associated hypothesis tests.

Table 3.

OSATS and procedure-specific checklist scores analysis results.

OSATS, and procedure-specific checklist scores analysis
	Group	AMean ± SD	BMean ± SD	CMean ± SD	p-values	Overall (test)	PairwiseA-B	PairwiseA-C	PairwiseB-C
OSATS	Pre-trainingu ± SD	13.40 ± 3.99	14.70 ± 5.61	10.20 ± 2.46	Pre-training Intergroup test	P = 0.0026 (Kruskal)	P = 0.9782	P = 0.0044	P = 0.0126
	Post-trainingu ± SD	25.90 ± 3.47	19.1 ± 7.52	11.05 ± 2.28	Post-training Intergroup test	P < 0.0001 (ANOVA)	P = 0.0001	P < 0.0001	P < 0.0001
	Intragroup (paired) tests	P < 0.0001(u-test)	P = 0.0002(t-test)	P < 0.0001(t-test)
	Group	AMean ± SD	BMean ± SD	CMean ± SD	p-values	Overall (test)	PairwiseA-B	PairwiseA-C	PairwiseB-C
Procedure-specific checklist	Pre-trainingu ± SD	6.85 ± 1.72	7.45 ± 2.52	6 ± 1.21	Pre-training Intergroup test	P = 0.0606(Kruskal)	P = 0.5803	P = 0.3397	P = 0.0590
	Post-trainingu ± SD	10.90 ± 1.37	8.75 ± 2.61	7.4 ± 1.31	Post-training Intergroup test	P < 0.0001(Kruskal)	P = 0.0166	P < 0.0001	P = 0.0747
	Intragroup (paired) tests	P = 0.0001(u-test)	P = 0.0011(t-test)	P < 0.0001(t-test)

Bold values indicate significant differences (p < 0.05).

The box and whisker charts in Figs. 2 and 3 show the distribution of the values measured in the pre-training and post-training stages of each scale. We see a higher median value post-training in all groups. However, group A tends to have the most significant difference between medians from the three groups in both scales.

Figure 2.

Pre- and post-training OSATS scores distribution.

Figure 3.

Pre- and post-training procedure-specific checklist scores distribution.

The average improvement in group A is the greater of all three groups, indicating a greater impact of the training. The increases in mean scores for Group A are OSATS: 12.5 points, and procedure-specific checklist: 4.05 points. Not only did Group A show significant improvement in their intragroup analysis, but they also show significant differences with Groups B and C for both scales in the post-training intergroup analysis. This indicates that Group A achieved significantly greater proficiency than Groups B and C at the post-training stage.

Group B also shows notable improvement, although of a slower magnitude than Group A. Increases in mean scores for Group B are OSATS: 4.4 points, and procedure-specific Checklist: 1.3 points. From the post-training intergroup analysis, it can be seen that Group B has significantly better scores than Group C for the OSATS scale but not in the specific procedure checklist. This indicates that although Group B fell short from Group A in both scales, they still achieved significantly better proficiency than Group C for the OSATS scale. However, their overall scores for the procedure-specific checklist are comparable to the scores from Group C.

Group C did show significant improvement in their intragroup analysis on both scales. Denoting that participating in the regular residency activities does improve proficiency. However, it must be noted that the magnitude of the difference between pre- and post-training scores is the lowest of all groups. Increases in mean scores for Group C are OSATS: 0.85 points, and procedure-specific Checklist: 1.4 points. From the post-training intergroup analysis, it is seen that Group C proficiency is significantly lower than Group A for both scales and Group B for one of the two scales.

A post-hoc statistical power analysis was conducted on the post-test hypothesis tests. Given the obtained evaluation tools, scores and the balanced groups with sample size n = 20 eleven out of the twelve post-tests obtained statistical power higher than the recommended 80 % (Ranging from 88 % to 100 %) and only one (Group B – Group C procedure-specific checklist post-test independent U-test) was below 80 % (with a statistical power of 52 %).

Discussion

The objective of this study was to analyze how a theoretical-practical workshop based on clinical simulation improves the practice of General Surgery residents in performing surgical knots necessary for their training. The OSATS, and procedure-specific checklist were used to evaluate the results obtained.

The collected data indicate that groups that participated in the theoretical-practical workshop (Groups A and B) obtained better results than Group C, which did not take the workshop or train with the assigned simulation model. In general, the groups that trained with the help of the workshop obtained statistically higher improvements compared to the group that did not do so. Group A has the most significant differences between pre- and post-training proficiency compared to the other groups, thanks to the theoretical-practical workshop, access to the low-cost model, and educational tools to train at home.

In the box and whisker plots comparing the pre-training and post-training results, we observe an upward shift in the scores of all groups for both scales, which suggests that the training has had a positive effect on all participants in all evaluates aspects.

From the intragroup analysis, it can be seen that all groups significantly improved their proficiency between their pre-training evaluation and their post-training evaluation. However, although all groups did improve, some improved more than others, this can be seen in the magnitude of the difference between the pre- and post-training scores.

These results indicate that the residents who attended the knot workshop and had access to the online platform in conjunction with the unsupervised practice at home had better results than those residents who, despite having attended the workshop, did not have said simulator to train at home. This highlights that although residency programs help aspiring surgeons to improve their skills, participating in additional workshops and specially having access to a low-cost simulator for home training can greatly accelerate the resident's skill development.

The advantages of using simulation models and workshops on surgical skills acquisition have been highlighted by other authors. Patel et al.¹⁸ performed a half-day workshop to promote interest among medical students in the surgical specialty and allow those interested in practicing basic surgical skills. All participants in their study demonstrated significant differences in their proficiency in suturing, knot tying and instrument identification by the end of the workshop similar to what we observed in our study. Scott et al.¹⁹ developed a proficiency-based knot-tying curriculum based on low-cost surgical boards, similar to the simulation model used in this study, for training one- and two-handed knots, participant of the validation of this curriculum significantly increased their surgical proficiency in the post-test compared to the pretest as did the sample analyzed in our study. Similarly, Empel et al.¹⁷ applied the OSATS scale to assess surgical resident's knot-tying skills after an advanced suturing course that consisted of one day of supervised training followed by six weeks of voluntary autonomous training at home using a knot-tying board, participant were evaluated before the supervised training, after the supervised training day ended, and after the six weeks autonomous training period, all participants significantly improved their OSATS score after the supervised training, and although the OSATS scores were slightly lower for some participants, the scores were not significantly different from those at the end of supervised training, demonstrating the autonomous training helped the participants to maintain their post-training proficiency. However in our study, we did not perform a follow-up on the improvement in participants’ proficiency after the workshop is shared.

The aforementioned studies along with our own show the importance of assisting surgical workshops besides the conventional residency activities and how the use of simulation models, even low-fidelity ones, can greatly improve suturing and knot-tying proficiency. However, although the mentioned studies show that participating in surgical workshops significantly improves surgical proficiency, there is no comparison in these studies of the magnitude of the difference between the proficiency of surgical trainees who attend the workshop compared to those who stay only with their residence activities, comparison which our study provides, further highlighting the importance of these kinds of workshops in medical institutions.

The main limitation of this study was the lack of homogeneity in the initial skills of the groups during the pre-training stage. Although no significant differences were found between Groups A and B at this stage, indicating that both groups have a similar degree of ability, significant differences were observed between Groups A-C and B-C. These differences suggest that Group C has a lower initial proficiency than the other two groups. Although we can still measure the magnitude of the difference in medians between pre-training and post-training evaluations of groups to know the impact of the two weeks of training with different settings and tools, having homogeneous groups would still have been better. Unfortunately, another limitation that prevented resampling to achieve homogeneity was the low enrollment rate of residents participating in the workshop. The low enrollment rate can be explained by the fact that residents who did not enroll in the study were guaranteed access to the low-cost simulator; meanwhile, enrolling in the study implied a chance to miss this benefit. In future studies, we will change the logistics of the simulator borrow so residents who fall in a group with no simulator can get it afterward to compensate and incentivize all residents to participate.

The study's objective was to evaluate the effect of having access to different learning tools, such as the theoretical-practical workshop and simulation models for continuous practice on learning basic surgical skills. The study showed that having access to teaching material to practice at home significantly improves learning the basic knots used in surgery.

The knot workshop, the online platform, and unsupervised practice at home significantly impacted residents' skills in tying surgical knots. The best results were observed in the group with access to both the workshop and practice tools at home.

Ethics

The study was approved by the ethics committee of the author's institution. All participants signed a letter of informed consent, and the study was carried out following the principles of the Declaration of Helsinki.