Interrater agreement of visual analysis in single-subject decisions: Quantitative review and analysis

Research output: Contribution to journalArticle

43 Citations (Scopus)

Abstract

The empirical literature examining the interrater agreement for visual analysis of single-subject data was examined using the technique of meta- analysis. Aggregated study results revealed an overall interrater agreement value of .58 for 14 studies representing a total of 789 raters. Additional analyses revealed that the interrater agreement values did not vary across raters based on previous experience or level of expertise. Characteristics of the graphs, including whether the data were hypothetical or real, or whether designs were the simple AB format or more complex (ABAB) designs, did not significantly influence the interrater agreement values. The only graphic characteristic related to increased interrater agreement was the presence of a trend (regression) line. Trend lines appeared to improve interrater agreement if the raters were familiar with their method of computation and interpretation. Simply including a trend line in the graphs produced only a marginal improvement in interrater agreement.

Original languageEnglish (US)
Pages (from-to)135-142
Number of pages8
JournalAmerican Journal on Mental Retardation
Volume98
Issue number1
StatePublished - 1993
Externally publishedYes

Fingerprint

Meta-Analysis
trend
Values
expertise
regression
interpretation
Visual Analysis
Interrater Agreement
experience
Raters
literature

ASJC Scopus subject areas

  • Rehabilitation
  • Education
  • Health Professions(all)

Cite this

@article{5280e7ae9f494211b641ab4bff552899,
title = "Interrater agreement of visual analysis in single-subject decisions: Quantitative review and analysis",
abstract = "The empirical literature examining the interrater agreement for visual analysis of single-subject data was examined using the technique of meta- analysis. Aggregated study results revealed an overall interrater agreement value of .58 for 14 studies representing a total of 789 raters. Additional analyses revealed that the interrater agreement values did not vary across raters based on previous experience or level of expertise. Characteristics of the graphs, including whether the data were hypothetical or real, or whether designs were the simple AB format or more complex (ABAB) designs, did not significantly influence the interrater agreement values. The only graphic characteristic related to increased interrater agreement was the presence of a trend (regression) line. Trend lines appeared to improve interrater agreement if the raters were familiar with their method of computation and interpretation. Simply including a trend line in the graphs produced only a marginal improvement in interrater agreement.",
author = "Kenneth Ottenbacher",
year = "1993",
language = "English (US)",
volume = "98",
pages = "135--142",
journal = "American Journal on Intellectual and Developmental Disabilities",
issn = "1944-7515",
publisher = "American Association on Intellectual and Developmental Disabilities",
number = "1",

}

TY - JOUR

T1 - Interrater agreement of visual analysis in single-subject decisions

T2 - Quantitative review and analysis

AU - Ottenbacher, Kenneth

PY - 1993

Y1 - 1993

N2 - The empirical literature examining the interrater agreement for visual analysis of single-subject data was examined using the technique of meta- analysis. Aggregated study results revealed an overall interrater agreement value of .58 for 14 studies representing a total of 789 raters. Additional analyses revealed that the interrater agreement values did not vary across raters based on previous experience or level of expertise. Characteristics of the graphs, including whether the data were hypothetical or real, or whether designs were the simple AB format or more complex (ABAB) designs, did not significantly influence the interrater agreement values. The only graphic characteristic related to increased interrater agreement was the presence of a trend (regression) line. Trend lines appeared to improve interrater agreement if the raters were familiar with their method of computation and interpretation. Simply including a trend line in the graphs produced only a marginal improvement in interrater agreement.

AB - The empirical literature examining the interrater agreement for visual analysis of single-subject data was examined using the technique of meta- analysis. Aggregated study results revealed an overall interrater agreement value of .58 for 14 studies representing a total of 789 raters. Additional analyses revealed that the interrater agreement values did not vary across raters based on previous experience or level of expertise. Characteristics of the graphs, including whether the data were hypothetical or real, or whether designs were the simple AB format or more complex (ABAB) designs, did not significantly influence the interrater agreement values. The only graphic characteristic related to increased interrater agreement was the presence of a trend (regression) line. Trend lines appeared to improve interrater agreement if the raters were familiar with their method of computation and interpretation. Simply including a trend line in the graphs produced only a marginal improvement in interrater agreement.

UR - http://www.scopus.com/inward/record.url?scp=0027274699&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027274699&partnerID=8YFLogxK

M3 - Article

C2 - 8373565

AN - SCOPUS:0027274699

VL - 98

SP - 135

EP - 142

JO - American Journal on Intellectual and Developmental Disabilities

JF - American Journal on Intellectual and Developmental Disabilities

SN - 1944-7515

IS - 1

ER -