Interrater reliability of therapists' judgments of graphed data

K. B. Harbst, Kenneth Ottenbacher, S. R. Harris

Research output: Contribution to journalArticle

50 Citations (Scopus)

Abstract

Increased emphasis on the use of single-subject designs in physical therapy research suggests the need to examine whether therapists can meaningfully interpret the results of such research as part of the clinical decision-making process. With this goal in mind, the interrater reliability of therapists to make visual judgments from graphed data that included a trend line was examined. Thirty therapists were presented with 24 graphs of single-subject data from AB (baseline-treatment) designs. Each graph included a trend line calculated using the split-middle method of trend estimation. The trend line was computed using the baseline data and then extended into the treatment phase to 'predict' patient performance. The analysis, using intraclass correlation coefficients (ICCs), revealed low interrater agreement, with ICC values ranging from .37 to .55 for the entire sample. Evidence is presented that the statistical backgrounds of some raters positively influenced interrater reliability. No statistically significant relationship was found between interrater agreement and visual components of the graphed data, such as changes in slope or variability.

Original languageEnglish (US)
Pages (from-to)107-115
Number of pages9
JournalPhysical Therapy
Volume71
Issue number2
StatePublished - 1991
Externally publishedYes

Fingerprint

Research
Therapeutics
Clinical Decision-Making

Keywords

  • Decision making
  • Research
  • Research design

ASJC Scopus subject areas

  • Orthopedics and Sports Medicine
  • Health Professions(all)
  • Physical Therapy, Sports Therapy and Rehabilitation

Cite this

Interrater reliability of therapists' judgments of graphed data. / Harbst, K. B.; Ottenbacher, Kenneth; Harris, S. R.

In: Physical Therapy, Vol. 71, No. 2, 1991, p. 107-115.

Research output: Contribution to journalArticle

Harbst, KB, Ottenbacher, K & Harris, SR 1991, 'Interrater reliability of therapists' judgments of graphed data', Physical Therapy, vol. 71, no. 2, pp. 107-115.
Harbst, K. B. ; Ottenbacher, Kenneth ; Harris, S. R. / Interrater reliability of therapists' judgments of graphed data. In: Physical Therapy. 1991 ; Vol. 71, No. 2. pp. 107-115.
@article{e9d279c2d7fd4db7b94d8c35b13e1a32,
title = "Interrater reliability of therapists' judgments of graphed data",
abstract = "Increased emphasis on the use of single-subject designs in physical therapy research suggests the need to examine whether therapists can meaningfully interpret the results of such research as part of the clinical decision-making process. With this goal in mind, the interrater reliability of therapists to make visual judgments from graphed data that included a trend line was examined. Thirty therapists were presented with 24 graphs of single-subject data from AB (baseline-treatment) designs. Each graph included a trend line calculated using the split-middle method of trend estimation. The trend line was computed using the baseline data and then extended into the treatment phase to 'predict' patient performance. The analysis, using intraclass correlation coefficients (ICCs), revealed low interrater agreement, with ICC values ranging from .37 to .55 for the entire sample. Evidence is presented that the statistical backgrounds of some raters positively influenced interrater reliability. No statistically significant relationship was found between interrater agreement and visual components of the graphed data, such as changes in slope or variability.",
keywords = "Decision making, Research, Research design",
author = "Harbst, {K. B.} and Kenneth Ottenbacher and Harris, {S. R.}",
year = "1991",
language = "English (US)",
volume = "71",
pages = "107--115",
journal = "Physical Therapy",
issn = "0031-9023",
publisher = "American Physical Therapy Association",
number = "2",

}

TY - JOUR

T1 - Interrater reliability of therapists' judgments of graphed data

AU - Harbst, K. B.

AU - Ottenbacher, Kenneth

AU - Harris, S. R.

PY - 1991

Y1 - 1991

N2 - Increased emphasis on the use of single-subject designs in physical therapy research suggests the need to examine whether therapists can meaningfully interpret the results of such research as part of the clinical decision-making process. With this goal in mind, the interrater reliability of therapists to make visual judgments from graphed data that included a trend line was examined. Thirty therapists were presented with 24 graphs of single-subject data from AB (baseline-treatment) designs. Each graph included a trend line calculated using the split-middle method of trend estimation. The trend line was computed using the baseline data and then extended into the treatment phase to 'predict' patient performance. The analysis, using intraclass correlation coefficients (ICCs), revealed low interrater agreement, with ICC values ranging from .37 to .55 for the entire sample. Evidence is presented that the statistical backgrounds of some raters positively influenced interrater reliability. No statistically significant relationship was found between interrater agreement and visual components of the graphed data, such as changes in slope or variability.

AB - Increased emphasis on the use of single-subject designs in physical therapy research suggests the need to examine whether therapists can meaningfully interpret the results of such research as part of the clinical decision-making process. With this goal in mind, the interrater reliability of therapists to make visual judgments from graphed data that included a trend line was examined. Thirty therapists were presented with 24 graphs of single-subject data from AB (baseline-treatment) designs. Each graph included a trend line calculated using the split-middle method of trend estimation. The trend line was computed using the baseline data and then extended into the treatment phase to 'predict' patient performance. The analysis, using intraclass correlation coefficients (ICCs), revealed low interrater agreement, with ICC values ranging from .37 to .55 for the entire sample. Evidence is presented that the statistical backgrounds of some raters positively influenced interrater reliability. No statistically significant relationship was found between interrater agreement and visual components of the graphed data, such as changes in slope or variability.

KW - Decision making

KW - Research

KW - Research design

UR - http://www.scopus.com/inward/record.url?scp=0025970836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0025970836&partnerID=8YFLogxK

M3 - Article

C2 - 1989006

AN - SCOPUS:0025970836

VL - 71

SP - 107

EP - 115

JO - Physical Therapy

JF - Physical Therapy

SN - 0031-9023

IS - 2

ER -