Does the choice of risk-adjustment model influence the outcome of surgeon-specific mortality analysis? A retrospective analysis of 14 637 patients under 31 surgeons
- University Hospital of South Manchester NHS Foundation Trust, Southmoor Road, Manchester M23 9LT, UK
- Mr Ben Bridgewater, University Hospital of South Manchester NHS Foundation Trust, Southmoor Road, Manchester M23 9LT, UK; ben.bridgewater{at}smuht.nwest.nhs.uk
- Accepted 9 October 2007
- Published Online First 1 November 2007
Abstract
Objectives: To compare implications of using the logistic EuroSCORE and a locally derived model when analysing individual surgeon mortality outcomes.
Design: Retrospective analysis of prospectively collected data.
Setting: All NHS hospitals undertaking adult cardiac surgery in northwest England.
Patients: 14 637 consecutive patients, April 2002 to March 2005.
Main outcome measures: We have compared the predictive ability of the logistic EuroSCORE (uncalibrated), the logistic EuroSCORE calibrated for contemporary performance and a locally derived logistic regression model. We have used each to create risk-adjusted individual surgeon mortality funnel plots to demonstrate high mortality outcomes.
Results: There were 458 (3.1%) deaths. The expected mortality and receiver operating characteristic (ROC) curve values were: uncalibrated EuroSCORE −5.8% and 0.80, calibrated EuroSCORE −3.1% and 0.80, locally derived model −3.1% and 0.82. The uncalibrated EuroSCORE plot showed one surgeon to have mortality above the northwest average, and no surgeon above the 95% control limit (CL). The calibrated EuroSCORE plot and the local model showed little change in surgeon ranking, but significant differences in identifying high mortality outcomes. Two of three surgeons above the 95% CL using the calibrated EuroSCORE revert to acceptable outcomes when the local model is applied but the finding is critically dependent on the calibration coefficient.
Conclusions: The uncalibrated EuroSCORE significantly overpredicted mortality and is not recommended. Instead, the EuroSCORE should be calibrated for contemporary performance. The differences demonstrated in defining high mortality outcomes when using a model built for purpose suggests that the choice of risk model is important when analysing surgeon mortality outcomes.
Footnotes
-
Competing interests: BB is a Society of Cardiothoracic Surgeons of GB and Ireland representative on the joint Society of Cardiothoracic Surgeons, Healthcare commission, Department of Health group defining national cardiac surgical audit and is on the executive committee of the SCTS. BB, AG, GJG, BF and MJ are all members of the steering group of the Northwest Regional Quality Improvement Programme in Cardiac Interventions.
-
Funding: Funding for the North West Quality Improvement Programme in Cardiac Interventions collaboration has been received from all primary care trusts in the north west of England. All authors were independent of the funding.
