The κ statistic in rehabilitation research: An examination

Tooth LR, Ottenbacher KJ. The κ statistic in rehabilitation research: an examination. 2004;85:1371-6. The number and sophistication of statistical procedures reported in medical rehabilitation research is increasing. Application of the principles and methods associated with evidence-based practice has contributed to the need for rehabilitation practitioners to understand quantitative methods in published articles. Outcomes measurement and determination of reliability are areas that have experienced rapid change during the past decade. In this study, distinctions between reliability and agreement are examined. Information is presented on analytical approaches for addressing reliability and agreement with the focus on the application of the κ statistic. The following assumptions are discussed: (1) κ should be used with data measured on a categorical scale, (2) the patients or objects categorized should be independent, and (3) the observers or raters must make their measurement decisions and judgments independently. Several issues related to using κ in measurement studies are described, including use of weighted κ, methods of reporting κ, the effect of bias and prevalence on κ, and sample size and power requirements for κ. The κ statistic is useful for assessing agreement among raters, and it is being used more frequently in rehabilitation research. Correct interpretation of the κ statistic depends on meeting the required assumptions and accurate reporting.