Use of the American College of Surgeons NSQIP surgical risk calculator for laparoscopic colectomy: How good is it and how can we improve it?

Background The American College of Surgeons NSQIP risk calculator was developed from multi-institutional clinical data to estimate preoperative risk. The impact of outliers has the potential to greatly affect predictions. Although the effect of outliers is minimized in large series, their impact on the individual provider or institution could be profound. No previous study has assessed the risk calculator for a single institution or provider, including outliers. Our goal was to evaluate the accuracy of the predicted outcomes at a single institution. Study Design Laparoscopic colectomies performed by two colorectal surgeons at a tertiary referral center were prospectively evaluated using the risk calculator. Predicted outcomes were compared with actual outcomes for length of stay (LOS), complications, return to the operating room, and death. Main outcomes measures were differences in actual vs predicted outcomes. Results One hundred and sixteen patients were included. Actual LOS was higher than predicted (mean ± SD 4.22 ± 5.49 days vs predicted 4.11 ± 1.18 days; p = 0.0001). Four outliers with multiple complications had an LOS >3 SDs from the mean. After removing these, observed LOS was significantly shorter than predicted (adjusted LOS mean ± SD 3.31 ± 2.30 days vs predicted 4.05 ± 1.14 days; p = 0.002). Occurrence of any complication was significantly lower than predicted (17.3% vs 19.4%; p = 0.05). Rates of major complications (13.2% vs 19.4%; p = 0.009) and surgical site infections (9.8% vs 11.8%; p = 0.006) were also significantly lower than predicted. There were no significant differences in death, urinary tract infection, renal failure, and reoperation rates. Conclusions Although the risk calculator was effective for evaluating average surgical-risk patients, it does not accurately predict outcomes in a small percentage of patients when one or more serious complications occur. Addition of surgeon- and patient-specific data via the American College of Surgeons case-logging system could better adjust for these areas.