2017 ISAKOS Biennial Congress ePoster #2402

 

The Fragility of Statistically Significant Findings from Randomized Trials in Sports Surgery: A Systematic Survey

Moin Khan, MD, MSc, FRCSC, Hamilton, ON CANADA
Nathan Evaniew, MD, Hamilton, ON CANADA
Mark Gichuru, BSc, Hamilton CANADA
Anthony Habib, MD, Hamilton CANADA
Olufemi R. Ayeni, MD, PhD, MSc, FRCSC, Hamilton, ON CANADA
Asheesh Bedi, MD, Ann Arbor, MI UNITED STATES
Michael Walsh, MD, Hamilton CANADA
PJ Devereaux, MD, FRCPC, Hamilton CANADA
Mohit Bhandari, MD, PhD, FRCSC, Hamilton, ON CANADA

McMaster University, Hamilton, Ontario, CANADA

FDA Status Not Applicable

Summary

Most statistically significant randomized trials in sports medicine and arthroscopic surgery are not robust.

Abstract

Introduction

High quality evidence-based orthopedic care relies on the generation and translation of robust research evidence. The Fragility Index is a novel method to evaluate the robustness of statistically significant findings from randomized control trials (RCTs). It is defined as the minimum number of patients in one arm of a trial that would have to change status from a non-event to an event to alter the results of the trial from statistically significant to non-significant. In this study we calculated the Fragility Index of statistically significant results from clinical trials in sports medicine and arthroscopic surgery to characterize the robustness of the RCTs in these fields.

Methods

We searched Medline and EMBASE for RCTs related to sports medicine and arthroscopic surgery from Jan 1st 2005 to Oct 30th 2015. Two reviewers independently assessed titles and abstracts for study eligibility, performed data extraction, and assessed risk of bias. The Fragility Index was calculated using Fisher’s exact test for all statistically significant dichotomous outcomes from parallel group RCT’s. Bivariate correlation was performed to evaluate associations between the Fragility Index and trial characteristics.

Results

Forty-eight RCT’s were included. The median sample size was 64 (Interquartile range, IQR 48.5 – 89.5) and the median total number of outcome events was 19 (IQR 10 – 27). The median Fragility Index was 2 (IQR 1-2.8), meaning changing two patients from a non-event to an event in the treatment arm changed the result to a statistically non-significant result, p=0.05.

Conclusion

Most statistically significant RCTs in sports medicine and arthroscopic surgery are not robust because their statistical significance can be reversed by only changing the outcome status on a few patients in one treatment group. Routine reporting of the Fragility Index may enhance clinicians’ ability to detect statistically significant trial results that should be viewed cautiously.