Statistical conclusion validity: An empirical analysis of multiplicity in mental retardation research

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

The statistical conclusion validity of mental retardation research was evaluated by reviewing 41 recently published articles in the American Journal on Mental Retardation and the Journal of Mental Deficiency Research. The analysis was focused on the incidence of Type I error as reflected by three different measures: the experiment-wise error rate, the error rate per experiment, and the percentage error rate. Values for each of these measures indicated that the probability of committing a Type I error was considerably greater than the traditionally assumed level of p less than .05, or 5%. The percentage error rate computed for the 41 articles (369 statistical tests) suggested that approximately 20% of the statistically significant results may be erroneous. Procedures to evaluate the impact of Type I errors in mental retardation research were presented and briefly discussed.

Original languageEnglish (US)
Pages (from-to)421-427
Number of pages7
JournalAmerican Journal on Mental Retardation
Volume95
Issue number4
StatePublished - 1991
Externally publishedYes

Fingerprint

Intellectual Disability
Research
statistical test
experiment
Mental Retardation
Multiplicity
Incidence
incidence
Values

ASJC Scopus subject areas

  • Rehabilitation
  • Education
  • Health Professions(all)

Cite this

@article{18a7cbd83fe14569a5f64f718a69b95e,
title = "Statistical conclusion validity: An empirical analysis of multiplicity in mental retardation research",
abstract = "The statistical conclusion validity of mental retardation research was evaluated by reviewing 41 recently published articles in the American Journal on Mental Retardation and the Journal of Mental Deficiency Research. The analysis was focused on the incidence of Type I error as reflected by three different measures: the experiment-wise error rate, the error rate per experiment, and the percentage error rate. Values for each of these measures indicated that the probability of committing a Type I error was considerably greater than the traditionally assumed level of p less than .05, or 5{\%}. The percentage error rate computed for the 41 articles (369 statistical tests) suggested that approximately 20{\%} of the statistically significant results may be erroneous. Procedures to evaluate the impact of Type I errors in mental retardation research were presented and briefly discussed.",
author = "Kenneth Ottenbacher",
year = "1991",
language = "English (US)",
volume = "95",
pages = "421--427",
journal = "American Journal on Intellectual and Developmental Disabilities",
issn = "1944-7515",
publisher = "American Association on Intellectual and Developmental Disabilities",
number = "4",

}

TY - JOUR

T1 - Statistical conclusion validity

T2 - An empirical analysis of multiplicity in mental retardation research

AU - Ottenbacher, Kenneth

PY - 1991

Y1 - 1991

N2 - The statistical conclusion validity of mental retardation research was evaluated by reviewing 41 recently published articles in the American Journal on Mental Retardation and the Journal of Mental Deficiency Research. The analysis was focused on the incidence of Type I error as reflected by three different measures: the experiment-wise error rate, the error rate per experiment, and the percentage error rate. Values for each of these measures indicated that the probability of committing a Type I error was considerably greater than the traditionally assumed level of p less than .05, or 5%. The percentage error rate computed for the 41 articles (369 statistical tests) suggested that approximately 20% of the statistically significant results may be erroneous. Procedures to evaluate the impact of Type I errors in mental retardation research were presented and briefly discussed.

AB - The statistical conclusion validity of mental retardation research was evaluated by reviewing 41 recently published articles in the American Journal on Mental Retardation and the Journal of Mental Deficiency Research. The analysis was focused on the incidence of Type I error as reflected by three different measures: the experiment-wise error rate, the error rate per experiment, and the percentage error rate. Values for each of these measures indicated that the probability of committing a Type I error was considerably greater than the traditionally assumed level of p less than .05, or 5%. The percentage error rate computed for the 41 articles (369 statistical tests) suggested that approximately 20% of the statistically significant results may be erroneous. Procedures to evaluate the impact of Type I errors in mental retardation research were presented and briefly discussed.

UR - http://www.scopus.com/inward/record.url?scp=0026027279&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0026027279&partnerID=8YFLogxK

M3 - Article

C2 - 2003911

AN - SCOPUS:0026027279

VL - 95

SP - 421

EP - 427

JO - American Journal on Intellectual and Developmental Disabilities

JF - American Journal on Intellectual and Developmental Disabilities

SN - 1944-7515

IS - 4

ER -