An examination of reliability in developmental research.

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

The purpose of this investigation was to examine quantitative methods used to determine reliability in developmental research. Procedures used to compute reliability estimates in 30 studies published in three developmental journals were examined. Four types of reliability studies were identified and analyzed. These included interrater reliability, stability (test-retest and intrarater reliability), equivalence reliability, and internal consistency. Interrater reliability investigations were the most frequently reported in the developmental literature reviewed (45%). The Pearson product moment correlation (r) was the most commonly reported reliability statistic. The findings reveal that researchers in developmental pediatrics frequently analyze reliability data using the Pearson product moment correlation and interpret the results as indicating consensus (agreement) among raters or across instruments. The Pearson product moment correlation (r) provides information on covariation among variables but does not indicate agreement. Thus, the findings suggest that developmental researchers may be misinterpreting the statistical results of reliability investigations. The argument is made that the intraclass correlation coefficient (ICC) is a more appropriate method of analysis when the purpose of the research is to examine consensus.

Original languageEnglish (US)
Pages (from-to)177-182
Number of pages6
JournalJournal of Developmental and Behavioral Pediatrics
Volume16
Issue number3
StatePublished - Jun 1995
Externally publishedYes

Fingerprint

Reproducibility of Results
Research
Research Personnel
Pediatrics

ASJC Scopus subject areas

  • Pediatrics, Perinatology, and Child Health
  • Behavioral Neuroscience
  • Psychology(all)
  • Developmental and Educational Psychology

Cite this

An examination of reliability in developmental research. / Ottenbacher, Kenneth.

In: Journal of Developmental and Behavioral Pediatrics, Vol. 16, No. 3, 06.1995, p. 177-182.

Research output: Contribution to journalArticle

@article{4d962d3c166d4c0ea4b5c8793462dae7,
title = "An examination of reliability in developmental research.",
abstract = "The purpose of this investigation was to examine quantitative methods used to determine reliability in developmental research. Procedures used to compute reliability estimates in 30 studies published in three developmental journals were examined. Four types of reliability studies were identified and analyzed. These included interrater reliability, stability (test-retest and intrarater reliability), equivalence reliability, and internal consistency. Interrater reliability investigations were the most frequently reported in the developmental literature reviewed (45{\%}). The Pearson product moment correlation (r) was the most commonly reported reliability statistic. The findings reveal that researchers in developmental pediatrics frequently analyze reliability data using the Pearson product moment correlation and interpret the results as indicating consensus (agreement) among raters or across instruments. The Pearson product moment correlation (r) provides information on covariation among variables but does not indicate agreement. Thus, the findings suggest that developmental researchers may be misinterpreting the statistical results of reliability investigations. The argument is made that the intraclass correlation coefficient (ICC) is a more appropriate method of analysis when the purpose of the research is to examine consensus.",
author = "Kenneth Ottenbacher",
year = "1995",
month = "6",
language = "English (US)",
volume = "16",
pages = "177--182",
journal = "Journal of Developmental and Behavioral Pediatrics",
issn = "0196-206X",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - An examination of reliability in developmental research.

AU - Ottenbacher, Kenneth

PY - 1995/6

Y1 - 1995/6

N2 - The purpose of this investigation was to examine quantitative methods used to determine reliability in developmental research. Procedures used to compute reliability estimates in 30 studies published in three developmental journals were examined. Four types of reliability studies were identified and analyzed. These included interrater reliability, stability (test-retest and intrarater reliability), equivalence reliability, and internal consistency. Interrater reliability investigations were the most frequently reported in the developmental literature reviewed (45%). The Pearson product moment correlation (r) was the most commonly reported reliability statistic. The findings reveal that researchers in developmental pediatrics frequently analyze reliability data using the Pearson product moment correlation and interpret the results as indicating consensus (agreement) among raters or across instruments. The Pearson product moment correlation (r) provides information on covariation among variables but does not indicate agreement. Thus, the findings suggest that developmental researchers may be misinterpreting the statistical results of reliability investigations. The argument is made that the intraclass correlation coefficient (ICC) is a more appropriate method of analysis when the purpose of the research is to examine consensus.

AB - The purpose of this investigation was to examine quantitative methods used to determine reliability in developmental research. Procedures used to compute reliability estimates in 30 studies published in three developmental journals were examined. Four types of reliability studies were identified and analyzed. These included interrater reliability, stability (test-retest and intrarater reliability), equivalence reliability, and internal consistency. Interrater reliability investigations were the most frequently reported in the developmental literature reviewed (45%). The Pearson product moment correlation (r) was the most commonly reported reliability statistic. The findings reveal that researchers in developmental pediatrics frequently analyze reliability data using the Pearson product moment correlation and interpret the results as indicating consensus (agreement) among raters or across instruments. The Pearson product moment correlation (r) provides information on covariation among variables but does not indicate agreement. Thus, the findings suggest that developmental researchers may be misinterpreting the statistical results of reliability investigations. The argument is made that the intraclass correlation coefficient (ICC) is a more appropriate method of analysis when the purpose of the research is to examine consensus.

UR - http://www.scopus.com/inward/record.url?scp=0029320967&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0029320967&partnerID=8YFLogxK

M3 - Article

C2 - 7560120

AN - SCOPUS:0029320967

VL - 16

SP - 177

EP - 182

JO - Journal of Developmental and Behavioral Pediatrics

JF - Journal of Developmental and Behavioral Pediatrics

SN - 0196-206X

IS - 3

ER -