Summary

Studies have shown limited levels of validation of arthroscopic simulators, especially in the area of construct and transfer validity. In our quasi-experimental study of arthroscopic simulator validation, we showed differences between an untrained student and a trained expert in arthroscopic surgery, giving it independent construct validity and an adequate appearance according to the experts.

Abstract

Introduction

Simulation has been taking place as a surgical learning method, allowing the acquisition of skills in an environment safe for students and patients. Despite being very frequent, arthroscopic surgery is not a procedure that residents perform regularly due to its complexity and risks for the patient. One of the most used arthroscopic simulators is the FAST. There are studies that show limited levels of validation of arthroscopic simulators, especially in the area of construct and transfer validity. The objective was to perform an independent study to validate the appearance and the construct of the FAST simulation module.

Methods

Quasi-experimental study. First-year residents of Orthopedics were recruited as novices; and arthroscopic surgeons (sports) from our institution's as experts. An arthroscopic training module was created on a remote asynchronous learning and feedback app (C1DO1®). The module had written and audiovisual content from the FAST simulator. Both students and experts underwent simulation in the FAST module, first with their dominant hand and then with their non-dominant hand. Both groups artrhroscopies were recorded. Performance was assessed using the Objective structured assessment of technical skill (OSATS, score 5-25) and the Arthroscopic Surgical Skill Evaluation Tool (ASSET, score 5-40). Both scores show better performance with higher scores. Videos were scored by a single another expert, blind to whom was performing the procedure. Construct validity was assessed by comparing performance between novices and experts. For appearance validity, a survey with questions on a Likert scale was applied. The Shapiro Wilk test was used to assess data distribution. Medians (minimum-maximum) were used with the Mann Whitney test to assess differences between groups. Statistical significance was determined using p<0.05 with a 95% confidence interval.

Results

A total of nine students and four experts were included. Students obtained a median score with the dominant hand significantly worse than score of experts (OSATS of 9 (5-13) vs 24 (23-25), p=0.01 respectively); as for the non-dominant hand, the score was also worse for students (OSATS of 8 (6-13) vs 24.5 (22-25), p=0.01 respectively). The median ASSET score was also significantly higher in experts than novices, both for the dominant hand (35 (27-37) vs 14 (7-20), p=0.01 respectively) and for the non-dominant hand (37.5 (32-38) vs 16 (13-20), p=0.01, respectively). The experts rated the module with a median of 4.5 (4-5) in its teaching value and 4.5 (4-5) in usability.

Conclusion

The FAST simulation model differentiates between an untrained student and a trained expert in arthroscopic surgery, giving it independent construct validity. Additionally, it has an adequate appearance according to experts.