Statistical Significance and Orthopaedic Traumatology

Abstract

Purpose of Study: A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. The present study explores how lowering the P value threshold would affect the interpretation of previously published trauma orthopaedic randomized clinical trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. Methods: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. Results: We identified 117 primary endpoints from 49 trials: 41 endpoints (35.0%) had a P value less than .05 and 76 (65.0%) had a P value greater than .05. Overall, 41.5% (17/41) of statistically significant primary endpoints were less than .005. Of the 117 primary endpoints, only 17 (14.5%) of the endpoints were less than .005, and would hold significance with the proposed threshold. Only 6.12% (3/49) of the included studies had all primary endpoints that met the new threshold of .005. Conclusion: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered.