Statistical Fragility And The Role Of P Values In The Hip Arthroscopy Literature
Robert L. Parisien MD1, David P. Trofa MD2, Michaela O’Connor BA2, Brock Knapp BA1, Emily J. Curry BA3, T. Sean Lynch MD2, Paul Tornetta III MD1 and Xinning Li MD1 1Boston University Medical Center, Boston, MA 2Columbia University Medical Center, New York, NY 3Boston University School of Public Health, Boston, MA
Background:
The purpose of this study was to critically examine the statistical stability of the published comparative studies in the hip arthroscopy literature.
Methods:
Comparative studies in the hip arthroscopy literature were analyzed in the Journal of Bone and Joint Surgery, American Journal of Sports Medicine, Arthroscopy: The Journal of Arthroscopic and Related Surgery, Knee Surgery-Sports Traumatology-Arthroscopy and the Journal of Hip Preservation Surgery between 2008-2018. The Fragility Index for each study outcome was determined by the number of event reversals required to change the P value to either greater or less than 0.05, thus changing study conclusions. The Fragility Quotient was determined by dividing the Fragility Index by the total population. Results Of the 4520 studies evaluated, 52 comparative studies met inclusion criteria consisting of a total of 150 outcome events. There were 7 randomized controlled trials and 45 non-randomized trials. Of the 150 total outcome events, 33 were initially reported as significant and 34 represented primary outcomes. The Fragility Index for the entire study was only 3.5 with an associated Fragility Quotient of 0.032. There was no difference appreciated between events initially reported as significant versus non-significant.
Conclusion:
Comparative studies in the hip arthroscopy literature may not be as stable as previously thought as the utilization of a single discreet P value cutoff has proven misleading. We therefore recommend adoption of the Fragility Index and Fragility Quotient, in conjunction with P value analysis, to aid in the evaluation of statistical stability in future comparative hip arthroscopy studies.
Back to 2019 Posters
