The Power of Replications and Replications of Power

The purpose of this paper is to examine the impact of low statistical power on the process of research replication. The traditional model of interpreting the success of replication efforts emphasizes the results of statistical significance testing. Serious errors in the interpretation of replicated studies may occur if the focus is on only statistical significance. Correct interpretation of the probability of replication must include a consideration of statistical power and related factors such as effect size and type 2 error rates. Examples are provided illustrating the importance of power, effect size, and type 2 error rates in developing a scientific consensus based on repeated experimental trials. The establishment of an empirical consensus is essential to scientific progress and may be improved by the use of aggregated effect size, where the focus is on determining the degree of replication rather than a dichotomous decision based on the results of statistical significance testing. Consensus may be delayed if investigators continue to focus on the statistical significance of individual studies and ignore the effect size and statistical power of the investigations they conduct.