The Power of Replications and Replications of Power

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

The purpose of this paper is to examine the impact of low statistical power on the process of research replication. The traditional model of interpreting the success of replication efforts emphasizes the results of statistical significance testing. Serious errors in the interpretation of replicated studies may occur if the focus is on only statistical significance. Correct interpretation of the probability of replication must include a consideration of statistical power and related factors such as effect size and type 2 error rates. Examples are provided illustrating the importance of power, effect size, and type 2 error rates in developing a scientific consensus based on repeated experimental trials. The establishment of an empirical consensus is essential to scientific progress and may be improved by the use of aggregated effect size, where the focus is on determining the degree of replication rather than a dichotomous decision based on the results of statistical significance testing. Consensus may be delayed if investigators continue to focus on the statistical significance of individual studies and ignore the effect size and statistical power of the investigations they conduct.

Original languageEnglish (US)
Pages (from-to)271-275
Number of pages5
JournalAmerican Statistician
Volume50
Issue number3
StatePublished - Aug 1996

Fingerprint

Effect Size
Statistical Significance
Statistical Power
Replication
Error Rate
Testing
Continue
Effect size
Statistical significance
Statistical power
Interpretation

Keywords

  • Experimental error
  • Research design
  • Statistical analysis

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability

Cite this

The Power of Replications and Replications of Power. / Ottenbacher, Kenneth.

In: American Statistician, Vol. 50, No. 3, 08.1996, p. 271-275.

Research output: Contribution to journalArticle

@article{ea8c7c2b97b243e5a4e8749431f1435f,
title = "The Power of Replications and Replications of Power",
abstract = "The purpose of this paper is to examine the impact of low statistical power on the process of research replication. The traditional model of interpreting the success of replication efforts emphasizes the results of statistical significance testing. Serious errors in the interpretation of replicated studies may occur if the focus is on only statistical significance. Correct interpretation of the probability of replication must include a consideration of statistical power and related factors such as effect size and type 2 error rates. Examples are provided illustrating the importance of power, effect size, and type 2 error rates in developing a scientific consensus based on repeated experimental trials. The establishment of an empirical consensus is essential to scientific progress and may be improved by the use of aggregated effect size, where the focus is on determining the degree of replication rather than a dichotomous decision based on the results of statistical significance testing. Consensus may be delayed if investigators continue to focus on the statistical significance of individual studies and ignore the effect size and statistical power of the investigations they conduct.",
keywords = "Experimental error, Research design, Statistical analysis",
author = "Kenneth Ottenbacher",
year = "1996",
month = "8",
language = "English (US)",
volume = "50",
pages = "271--275",
journal = "American Statistician",
issn = "0003-1305",
publisher = "American Statistical Association",
number = "3",

}

TY - JOUR

T1 - The Power of Replications and Replications of Power

AU - Ottenbacher, Kenneth

PY - 1996/8

Y1 - 1996/8

N2 - The purpose of this paper is to examine the impact of low statistical power on the process of research replication. The traditional model of interpreting the success of replication efforts emphasizes the results of statistical significance testing. Serious errors in the interpretation of replicated studies may occur if the focus is on only statistical significance. Correct interpretation of the probability of replication must include a consideration of statistical power and related factors such as effect size and type 2 error rates. Examples are provided illustrating the importance of power, effect size, and type 2 error rates in developing a scientific consensus based on repeated experimental trials. The establishment of an empirical consensus is essential to scientific progress and may be improved by the use of aggregated effect size, where the focus is on determining the degree of replication rather than a dichotomous decision based on the results of statistical significance testing. Consensus may be delayed if investigators continue to focus on the statistical significance of individual studies and ignore the effect size and statistical power of the investigations they conduct.

AB - The purpose of this paper is to examine the impact of low statistical power on the process of research replication. The traditional model of interpreting the success of replication efforts emphasizes the results of statistical significance testing. Serious errors in the interpretation of replicated studies may occur if the focus is on only statistical significance. Correct interpretation of the probability of replication must include a consideration of statistical power and related factors such as effect size and type 2 error rates. Examples are provided illustrating the importance of power, effect size, and type 2 error rates in developing a scientific consensus based on repeated experimental trials. The establishment of an empirical consensus is essential to scientific progress and may be improved by the use of aggregated effect size, where the focus is on determining the degree of replication rather than a dichotomous decision based on the results of statistical significance testing. Consensus may be delayed if investigators continue to focus on the statistical significance of individual studies and ignore the effect size and statistical power of the investigations they conduct.

KW - Experimental error

KW - Research design

KW - Statistical analysis

UR - http://www.scopus.com/inward/record.url?scp=0030336428&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030336428&partnerID=8YFLogxK

M3 - Article

VL - 50

SP - 271

EP - 275

JO - American Statistician

JF - American Statistician

SN - 0003-1305

IS - 3

ER -