Reliability analysis in therapeutic research: practice and procedures.

K. J. Ottenbacher, S. D. Tomchek

Research output: Contribution to journalArticle

50 Scopus citations

Abstract

Twenty studies examining the reliability of assessment devices and outcome measures in therapeutic research were reviewed and analyzed. The 20 investigations contained 215 quantitative reliability values published in either the American Journal of Occupational Therapy or Physical Therapy during the past 5 years. The reliability studies were classified as interrater, intrarater, test-retest, or internal consistency. Examination of interrater reliability accounted for 41% of all reported reliability values. Studies published in Physical Therapy were more likely to be concerned with test-retest reliability, whereas studies published in the American Journal of Occupational Therapy more often focused on interrater reliability. Examination of the data revealed that the intraclass correlation coefficient (ICC) was the most frequently reported estimate of reliability, accounting for 57% of all reported reliability coefficients. Further review of the results indicated that Pearson product-moment correlations and percentage of agreement indexes accounted for 22% of all reliability values reported in the studies examined. The Pearson product-moment correlation measures association or covariation among variables, but not agreement, and percentage agreement indexes do not correct for chance agreement. The argument is made that product-moment correlations and percentage agreement indexes are inadequate measures of interrater, intrarater or test-retest agreement. They should be used and interpreted with caution.

Original languageEnglish (US)
Pages (from-to)10-16
Number of pages7
JournalThe American journal of occupational therapy. : official publication of the American Occupational Therapy Association
Volume47
Issue number1
DOIs
StatePublished - Jan 1993
Externally publishedYes

ASJC Scopus subject areas

  • Occupational Therapy

Fingerprint Dive into the research topics of 'Reliability analysis in therapeutic research: practice and procedures.'. Together they form a unique fingerprint.

  • Cite this