Logistic, additive or historical: is EuroSCORE an appropriate model for comparing individual surgeons’ performance?

B E Keogh

Correspondence to:
Professor Bruce E Keogh
The Heart Hospital, Suite 501, 16–18 Westmoreland Street, London W1G 8PH, UK; bruce.keogh{at}uclh.org

Both versions of the EuroSCORE, the complex logistic and the simpler additive, have been used for comparing institutional performance, but they are no longer robust enough to be used for comparing individual surgeons. It is now time to tighten the standard and add additional data to obtain a more holistic picture of the quality of cardiac surgical care in the UK.

In April 2006, the Society for Cardiothoracic Surgery in Great Britain and Ireland, in conjunction with the Healthcare Commission, published cardiac surgical results for individual surgical units and, in some cases for surgeons, on the Healthcare Commission’s website, using logistic EuroSCORE (European System for Cardiac Operative Risk Evaluation) as the risk adjustment model.¹ Publication of these data was well received by the media and the public, firstly, because of its transparency and, secondly, because it indicated that all units and surgeons were performing within the acceptable limits defined by EuroSCORE.

However, in this issue of Heart, Bhatti et al² have demonstrated that in the northwest of England, the EuroSCORE tends to overpredict operative mortality by a factor of 2. This raises the issue as to whether or not the EuroSCORE remains a good model for risk adjustment in cardiac surgery. The EuroSCORE was developed during the 1990s, with the aim of devising a simple and pragmatic risk adjustment model based on easily collectable data that could be used for both preoperative risk assessment and post-hoc comparison of surgical performance based on contemporary European data.³ EuroSCORE has two versions: a complex logistic version and a simpler additive version. Both versions have been shown in a number of publications to provide a good model for risk prediction in cardiac surgery at international,^4,5 hospital or unit level.⁶ The simultaneous publication of the article by Bhatti et al and the Healthcare Commission survival statistics raises two questions. Firstly, is the EuroSCORE still sufficiently contemporary to be credible? Secondly, is it robust enough for risk adjustment when comparing the performance of individual surgeons rather than hospitals?

Bhatti et al’s finding is endorsed by reworking of data underpinning the Healthcare Commission website and the Australian national database in Victoria.⁷ Although there was emerging evidence that the EuroSCORE was drifting towards overprediction in 2003,⁸ the magnitude of overperformance by UK surgeons could not have been predicted. We continued with the application of logistic EuroSCORE because it is the only validated European model, it is widely used in UK units and its use for this purpose had been agreed to before the collection and merging of national data.⁹ The fact that EuroSCORE overpredicts mortality is a reflection of sustained improvements in cardiac care nationally. Similar national data from other European countries are not available.¹⁰

So how does this relate to the use of EuroSCORE for comparison of individual surgeons’ performance? The most important predictors of outcome are operation type, urgency and age. Also, most units in the UK have broadly similar proportions of coronary and valve operations, but that variation is much greater between individual surgeons because of special interests such as off-pump coronary surgery, repairative mitral valve surgery, aortic surgery, and so forth. So when comparing units the type of operation is less important in the risk adjustment hierarchy than when comparing individual surgeons. This is where EuroSCORE fails. In 2003 in the UK, the in-hospital mortalities for aortic valve replacement (3.2%), mitral valve replacement (5.4%), mitral valve repair (1.3%), combined aortic and mitral valve operations (4.3%) and combined mitral and tricuspid operations (9.1%) went up to 6.8%, 8.6%, 6.2%, 11.8% and 14%, respectively, when combined with coronary surgery, but in EuroSCORE the influence of each of these operations on the final score is the same. So with similar age and comorbidities, a triple valve replacement with coronary grafts scores the same as an atrial septal defect repair.

Thus, a new algorithm is clearly required for enabling comparative performance in the UK. There are several options. The first is simply to recalibrate EuroSCORE by dividing by two. While attractive in its simplicity, this ignores the changing impact of different risk factors over time^11,12 and the inability of EuroSCORE to provide a discriminatory weighting for different operations. The second option is to use an international model and this is almost certainly where we will end up. However, the most widely used models remain proprietary¹³ or would require a change in our dataset.¹⁴ Such an approach would also raise issues of confidence in international data quality. The final and most pragmatic option is to refine and employ operation-specific risk models based on contemporary UK data, such as the UK Bayes model for coronary surgery⁸ and a generic valve model.¹⁵ These could be updated on an annual basis, as is the New York registry.

The approach to risk adjustment in terms of predictive accuracy, discrimination and frequency of recalibration is determined by what we are trying to achieve. There are three models encapsulating the reasons for publishing this sort of data at an institutional or an individual level.¹⁶

The first is a public accountability model which sees public disclosure as a public responsibility, irrespective of the consequences whereby release of the data, in conjunction with appropriate education and subsequent informed debate, will help clarify important societal issues and also improve standards.

The second is a market-oriented model, which assumes that the provision of comparative data will allow informed and willing consumers to drive quality improvement through selective purchasing or utilisation behaviour. To make valid and fair comparisons the data need to be standardised.

Finally, a professionally driven model assumes healthcare professionals have a desire to monitor and improve standards. This is generally motivated by a desire to retain autonomy in the face of greater governmental regulation. Providing data on variations aids this process, and publication increases provider responsiveness. The data act as a catalyst to identify and solve problems, and publication turns up the heat.

These models are not mutually exclusive, and the publication of cardiac surgical results in the UK has been driven to a variable extent by all three models. However, the choice of logistic EuroSCORE and the mode of presentation on the Healthcare Commission website was primarily to demonstrate compliance with a widely accepted European standard and not to provide graduated, categorical data to facilitate ranking of surgeons under the guise of "patient choice".

The venture has been a success. It is now time for us to tighten the standard and add additional data on hospital facilities, processes and other outcomes relating to morbidity, such as resternotomy and length of stay, in order to paint a more holistic picture of cardiac surgery in the UK.

What does this signal for cardiology and other specialties? The New York State Department of Health has published an operator-specific angioplasty report since 1995.¹⁴ The Chief Medical Officer’s consultation on revalidation,¹⁷ coupled with the desire of the Department of Health in England to see publication of unit-specific, specialty-based outcomes to underpin patient choice, will bring urgent pressure to bear on other interventional specialties, including cardiology, to identify useful outcome measures that can be risk adjusted.

Published Online First 27 September 2006

Competing interests: BEK is President of the Society for Cardiothoracic Surgery in Great Britain and Ireland, and a Commissioner on the Healthcare Commission.

1. Healthcare Commission. Heart surgery in Great Britain. http://heartsurgery.healthcarecommission.org.uk/ (accessed 20 Sep 2006).
2. Bhatti F, Grayson AD, Grotte G, et al. The logistic EuroSCORE in cardiac surgery: how well does it predict operative risk? Heart 2006;92:1817–20.[Abstract/Free Full Text]
3. Nashef SA, Roques F, Michel P, et al. European system for cardiac operative risk evaluation (EuroSCORE). Eur J Cardiothorac Surg 1999;16:9–13.[Abstract/Free Full Text]
4. Roques F, Nashef S, Michel P, et al. Does EuroSCORE work in individual European countries? Eur J Cardiothorac Surg 2000;18:27–30.[Abstract/Free Full Text]
5. Nashef SAM, Roques F, Hammill BG, et al. Validation of European System for Cardiac Operative Risk Evaluation (EuroSCORE) in North American cardiac surgery. Eur J Cardiothorac Surg 2002;22:101–5.[Abstract/Free Full Text]
6. Gogbashian A, Sedrakyan A, Treasure T. EuroSCORE: a systematic review of international performance. Eur J Cardiothorac Surg 2004;25:695–700.[Abstract/Free Full Text]
7. Yap C-H, Reid C, Yii M, et al. Validation of the EuroSCORE model in Australia. Eur J Cardiothorac Surg 2006;29:441–6.[Abstract/Free Full Text]
8. Keogh B, Kinsman R. Fifth National Adult Cardiac Surgical Database Report 2003: improving outcomes for patients. Reading, MA: Dendrite Clinical Systems, 2004 (ISBN 1-903968-09-7).
9. Petrou M, Roques F, Sharples L, et al. Risk model of choice for coronary surgery in the United Kingdom. Heart 2003;89:98–9.[Free Full Text]
10. Keogh B, Kinsman R. Second European Adult Cardiac Surgical Database Report. Reading, MA: Dendrite Clinical Systems, 2004 (ISBN 1-903968-13-5).
11. Gao D, Grunwald GK, Rumsfeld JS, et al. Time-varying risk factors for long-term mortality after coronary artery bypass graft surgery. Ann Thorac Surg 2006;81:793–9.[Abstract/Free Full Text]
12. Edwards FH. Evolution of risk factors in coronary bypass surgery. In: Keogh B, Kinsman R, eds. Fifth National Adult Cardiac Surgical Database Report 2003: improving outcomes for patients. Reading, MA: Dendrite Clinical Systems, 2004, 211–12 (ISBN 1-903968-09-7).
13. Shroyer AL, Coombs LP, Peterson ED, et al. The Society of Thoracic Surgeons: 30-day operative mortality and morbidity risk models. Ann Thorac Surg 2003;75:1856–65.[Abstract/Free Full Text]
14. New York State Department of Health. Heart disease. http://www.health.state.ny.us/nysdoh/heart/heart_disease.htm (accessed 20 Sep 2006).
15. Ambler G, Omar R, Royston P, et al. A generic, simple risk stratification model for heart valve surgery. Circulation 2005;112:224–31.[Abstract/Free Full Text]
16. Marshall M, Shekelle P, Brook R, et al. Dying to know. Public release of information about quality of healthcare. Nuffield Trust Series No: 12, Nuffield Trust, 2000 (ISBN 0-8330-2891-X).
17. Department of Health. Good doctors, safer patients. Proposals to strengthen the system to assure and improve the performance of doctors and to protect the safety of patients. A report by the Chief Medical Officer. London: Department of Health, 2006, http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticle/fs/en?CONTENT_ID=4137232&chk=KW63va or http://www.dh.gov.uk (accessed 20 Sep 2006).

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

• Granton, J., Cheng, D. (2008). Risk Stratification Models for Cardiac Surgery. SEMIN CARDIOTHORAC VASC ANESTH 12: 167-174 [Abstract]