Comparison of visual inspection and statistical analysis of single- subject data in rehabilitation research

Candace D. Bobrovitz, Kenneth Ottenbacher

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Single-subject designs are being advocated to conduct outcome research in rehabilitation environments. The methods provide an alternative to traditional designs based on statistical comparisons across groups. Data analysis in single subject research does not rely on statistical hypothesis testing of responses collected from a sample of subjects. Instead, visual inspection of patient responses graphed over time is the usual method of data analysis in single-subject research. This study examined the agreement between visual analysis and statistical tests of single-subject data for 42 hypothetical single-subject graphs. Specially constructed graphs allowed the systematic manipulation of different treatment effect sizes across a commonly used single-subject design. Thirty-two rehabilitation and health care providers rated each of the 42 graphs to determine whether a clinically significant treatment effect existed across the phases of the designs. Data analysis focused on two questions: (1) How much agreement was there between visual judgments and the results of statistical tests? and (2) What level of treatment effect was required to produce a finding of visual versus statistical significance? The agreement between visual analysis and statistical significance was high (86%). The sensitivity of visual inferences compared with statistical test results was 0.84, specificity was 0.88, and positive predictive value was 0.91. Both visual and statistical procedures were sensitive to medium and large treatment effects in the 42 single- subject graphs examined in this study.

Original languageEnglish (US)
Pages (from-to)94-102
Number of pages9
JournalAmerican Journal of Physical Medicine and Rehabilitation
Volume77
Issue number2
DOIs
StatePublished - Mar 1998

Fingerprint

Rehabilitation
Therapeutics
Research
Health Personnel
Outcome Assessment (Health Care)
Rehabilitation Research

Keywords

  • Data analysis
  • Statistical tests
  • Visual inspection

ASJC Scopus subject areas

  • Rehabilitation
  • Health Professions(all)
  • Orthopedics and Sports Medicine
  • Physical Therapy, Sports Therapy and Rehabilitation

Cite this

@article{9c062ed81d754717925e499476455ee9,
title = "Comparison of visual inspection and statistical analysis of single- subject data in rehabilitation research",
abstract = "Single-subject designs are being advocated to conduct outcome research in rehabilitation environments. The methods provide an alternative to traditional designs based on statistical comparisons across groups. Data analysis in single subject research does not rely on statistical hypothesis testing of responses collected from a sample of subjects. Instead, visual inspection of patient responses graphed over time is the usual method of data analysis in single-subject research. This study examined the agreement between visual analysis and statistical tests of single-subject data for 42 hypothetical single-subject graphs. Specially constructed graphs allowed the systematic manipulation of different treatment effect sizes across a commonly used single-subject design. Thirty-two rehabilitation and health care providers rated each of the 42 graphs to determine whether a clinically significant treatment effect existed across the phases of the designs. Data analysis focused on two questions: (1) How much agreement was there between visual judgments and the results of statistical tests? and (2) What level of treatment effect was required to produce a finding of visual versus statistical significance? The agreement between visual analysis and statistical significance was high (86{\%}). The sensitivity of visual inferences compared with statistical test results was 0.84, specificity was 0.88, and positive predictive value was 0.91. Both visual and statistical procedures were sensitive to medium and large treatment effects in the 42 single- subject graphs examined in this study.",
keywords = "Data analysis, Statistical tests, Visual inspection",
author = "Bobrovitz, {Candace D.} and Kenneth Ottenbacher",
year = "1998",
month = "3",
doi = "10.1097/00002060-199803000-00002",
language = "English (US)",
volume = "77",
pages = "94--102",
journal = "American Journal of Physical Medicine and Rehabilitation",
issn = "0894-9115",
publisher = "Lippincott Williams and Wilkins",
number = "2",

}

TY - JOUR

T1 - Comparison of visual inspection and statistical analysis of single- subject data in rehabilitation research

AU - Bobrovitz, Candace D.

AU - Ottenbacher, Kenneth

PY - 1998/3

Y1 - 1998/3

N2 - Single-subject designs are being advocated to conduct outcome research in rehabilitation environments. The methods provide an alternative to traditional designs based on statistical comparisons across groups. Data analysis in single subject research does not rely on statistical hypothesis testing of responses collected from a sample of subjects. Instead, visual inspection of patient responses graphed over time is the usual method of data analysis in single-subject research. This study examined the agreement between visual analysis and statistical tests of single-subject data for 42 hypothetical single-subject graphs. Specially constructed graphs allowed the systematic manipulation of different treatment effect sizes across a commonly used single-subject design. Thirty-two rehabilitation and health care providers rated each of the 42 graphs to determine whether a clinically significant treatment effect existed across the phases of the designs. Data analysis focused on two questions: (1) How much agreement was there between visual judgments and the results of statistical tests? and (2) What level of treatment effect was required to produce a finding of visual versus statistical significance? The agreement between visual analysis and statistical significance was high (86%). The sensitivity of visual inferences compared with statistical test results was 0.84, specificity was 0.88, and positive predictive value was 0.91. Both visual and statistical procedures were sensitive to medium and large treatment effects in the 42 single- subject graphs examined in this study.

AB - Single-subject designs are being advocated to conduct outcome research in rehabilitation environments. The methods provide an alternative to traditional designs based on statistical comparisons across groups. Data analysis in single subject research does not rely on statistical hypothesis testing of responses collected from a sample of subjects. Instead, visual inspection of patient responses graphed over time is the usual method of data analysis in single-subject research. This study examined the agreement between visual analysis and statistical tests of single-subject data for 42 hypothetical single-subject graphs. Specially constructed graphs allowed the systematic manipulation of different treatment effect sizes across a commonly used single-subject design. Thirty-two rehabilitation and health care providers rated each of the 42 graphs to determine whether a clinically significant treatment effect existed across the phases of the designs. Data analysis focused on two questions: (1) How much agreement was there between visual judgments and the results of statistical tests? and (2) What level of treatment effect was required to produce a finding of visual versus statistical significance? The agreement between visual analysis and statistical significance was high (86%). The sensitivity of visual inferences compared with statistical test results was 0.84, specificity was 0.88, and positive predictive value was 0.91. Both visual and statistical procedures were sensitive to medium and large treatment effects in the 42 single- subject graphs examined in this study.

KW - Data analysis

KW - Statistical tests

KW - Visual inspection

UR - http://www.scopus.com/inward/record.url?scp=0032032554&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0032032554&partnerID=8YFLogxK

U2 - 10.1097/00002060-199803000-00002

DO - 10.1097/00002060-199803000-00002

M3 - Article

C2 - 9558008

AN - SCOPUS:0032032554

VL - 77

SP - 94

EP - 102

JO - American Journal of Physical Medicine and Rehabilitation

JF - American Journal of Physical Medicine and Rehabilitation

SN - 0894-9115

IS - 2

ER -