Regression coefficient-based scoring system should be used to assign weights to the risk index

Hemalkumar Mehta, Vinay Mehta, Cynthia J. Girman, Deepak Adhikari, Michael L. Johnson

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Objective: Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient-based and risk ratio-based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting: This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient-based and risk ratio-based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results: Regression coefficient-based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio-based models (HR/Charlson, HR/Johnson). Regression coefficient-based CCS reclassified more number of people into the correct strata (NRI range, 9.02-10.04) compared to risk ratio-based CCS (NRI range, 8.14-8.22). Conclusion: Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio-based scoring systems. Researchers should use a regression coefficient-based scoring system to develop a risk index, which is theoretically correct.

Original languageEnglish (US)
JournalJournal of Clinical Epidemiology
DOIs
StateAccepted/In press - 2016

Fingerprint

Comorbidity
Odds Ratio
Weights and Measures
Mortality
Proportional Hazards Models
Cohort Studies
Retrospective Studies
Research Personnel
Research

Keywords

  • Charlson comorbidity score
  • Regression coefficient
  • Risk index
  • Risk ratio
  • Scoring algorithm
  • Scoring system

ASJC Scopus subject areas

  • Epidemiology

Cite this

Regression coefficient-based scoring system should be used to assign weights to the risk index. / Mehta, Hemalkumar; Mehta, Vinay; Girman, Cynthia J.; Adhikari, Deepak; Johnson, Michael L.

In: Journal of Clinical Epidemiology, 2016.

Research output: Contribution to journalArticle

Mehta, Hemalkumar ; Mehta, Vinay ; Girman, Cynthia J. ; Adhikari, Deepak ; Johnson, Michael L. / Regression coefficient-based scoring system should be used to assign weights to the risk index. In: Journal of Clinical Epidemiology. 2016.
@article{f464380e3a054f8cba400a4aca4fc972,
title = "Regression coefficient-based scoring system should be used to assign weights to the risk index",
abstract = "Objective: Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient-based and risk ratio-based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting: This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient-based and risk ratio-based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results: Regression coefficient-based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio-based models (HR/Charlson, HR/Johnson). Regression coefficient-based CCS reclassified more number of people into the correct strata (NRI range, 9.02-10.04) compared to risk ratio-based CCS (NRI range, 8.14-8.22). Conclusion: Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio-based scoring systems. Researchers should use a regression coefficient-based scoring system to develop a risk index, which is theoretically correct.",
keywords = "Charlson comorbidity score, Regression coefficient, Risk index, Risk ratio, Scoring algorithm, Scoring system",
author = "Hemalkumar Mehta and Vinay Mehta and Girman, {Cynthia J.} and Deepak Adhikari and Johnson, {Michael L.}",
year = "2016",
doi = "10.1016/j.jclinepi.2016.03.031",
language = "English (US)",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Regression coefficient-based scoring system should be used to assign weights to the risk index

AU - Mehta, Hemalkumar

AU - Mehta, Vinay

AU - Girman, Cynthia J.

AU - Adhikari, Deepak

AU - Johnson, Michael L.

PY - 2016

Y1 - 2016

N2 - Objective: Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient-based and risk ratio-based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting: This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient-based and risk ratio-based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results: Regression coefficient-based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio-based models (HR/Charlson, HR/Johnson). Regression coefficient-based CCS reclassified more number of people into the correct strata (NRI range, 9.02-10.04) compared to risk ratio-based CCS (NRI range, 8.14-8.22). Conclusion: Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio-based scoring systems. Researchers should use a regression coefficient-based scoring system to develop a risk index, which is theoretically correct.

AB - Objective: Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient-based and risk ratio-based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting: This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient-based and risk ratio-based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results: Regression coefficient-based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio-based models (HR/Charlson, HR/Johnson). Regression coefficient-based CCS reclassified more number of people into the correct strata (NRI range, 9.02-10.04) compared to risk ratio-based CCS (NRI range, 8.14-8.22). Conclusion: Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio-based scoring systems. Researchers should use a regression coefficient-based scoring system to develop a risk index, which is theoretically correct.

KW - Charlson comorbidity score

KW - Regression coefficient

KW - Risk index

KW - Risk ratio

KW - Scoring algorithm

KW - Scoring system

UR - http://www.scopus.com/inward/record.url?scp=84977492400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84977492400&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2016.03.031

DO - 10.1016/j.jclinepi.2016.03.031

M3 - Article

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -