This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on ISI (20)  • Contact me when this article is cited  Related Content  • Similar articles in this journal  Topic Collections  • Evidence-Based Medicine  • Randomized Controlled Trial  • Alert me on articles by topic

Equivalency or Error?

Justin B. Dimick, MD; Marie Diener-West, PhD; Pamela A. Lipsett, MD

Arch Surg. 2001;136:796-800.

Hypothesis  We hypothesized that review of randomized controlled clinical trials (RCTs) with nonstatistically significant or "negative" results published in the surgical literature do not have appropriate statistical power to demonstrate equivalency between treatment arms.

Data Sources and Study Selection  The MEDLINE database was searched to obtain reports of all RCTs with negative results published in 3 surgical journals from 1988 to 1998. Manual review of one year (1997) of publications for each journal was performed to validate our search strategy. Equivalency was evaluated using the Two One-Sided Tests Procedure and post hoc power calculations.

Data Synthesis  Ninety reports of RCTs with negative results were identified in the surgical literature between 1988 and 1998. The manual review of 1997 showed a 100% retrieval rate for our search strategy. After applying the Two One-Sided Tests Procedure, 35 reports (39%) met the criteria for demonstrating equivalency. The other 55 reports (61%) contained at least a 10% absolute difference in the 90% confidence interval of . Using the power calculation method, only 22 (24%) articles had a power greater than .80 to detect a 50% difference in therapeutic effect. Only 29% of the reports included a formal sample size calculation and these studies were more likely to demonstrate equivalency than those without a sample size estimate (P<.01).

Conclusions  Many reports from negative RCTs published in the surgical literature lack sufficient statistical power to establish that clinically important differences are not present. Surgeons should perform appropriate sample size calculations when designing RCTs and recognize the utility of confidence intervals when reporting negative results.

From the Departments of Surgery (Drs Dimick and Lipsett) and Anesthesiology/Critical Care Medicine (Dr Lipsett), Johns Hopkins University School of Medicine; and the Department of Biostatistics (Dr Diener-West), Johns Hopkins University School of Hygiene and Public Health, Baltimore, Md.

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Statistical Power of Negative Randomized Controlled Trials Presented at American Society for Clinical Oncology Annual Meetings
Bedard et al.
JCO 2007;25:3482-3487.
ABSTRACT | FULL TEXT  

Meta-analysis in Surgery--Invited Critique
Chang
Arch Surg 2006;141:1131-1131.
FULL TEXT  

Randomized controlled trials in surgery: Comic opera no more?
Kassell and Dumont
J. Thorac. Cardiovasc. Surg. 2006;132:243-244.
FULL TEXT  

Reporting of Noninferiority and Equivalence Randomized Trials: An Extension of the CONSORT Statement
Piaggio et al.
JAMA 2006;295:1152-1160.
ABSTRACT | FULL TEXT  

Challenges in Systematic Reviews of Therapeutic Devices and Procedures
Hartling et al.
ANN INTERN MED 2005;142:1100-1111.
ABSTRACT | FULL TEXT  

Interpreting Negative Results: Postpartum Interview Position Not Associated With Improved Outcomes
Mabry et al.
Arch Pediatr Adolesc Med 2003;157:333-335.
FULL TEXT