Send letter to journal:
Re: What Constitutes a Simple and Useful Risk Score?

What Constitutes a Simple and Useful Risk Score?

We read with interest the study by Dr. Gale and colleagues.[1] We fully agree with the authors that age, heart rate, and systolic blood pressure are among the most powerful prognosticators in acute coronary syndromes (ACS).[2,3] However, we believe that several key issues deserve clarification.

First, by dichotomizing continuous variables such as creatinine and Killip class (e.g. Killip class II and IV would be considered equivalent as “cardiac failure”), substantial prognostic information would be lost, so that performance of risk scores consisting of such variables would be underestimated. Furthermore, by not using the coefficients derived from the original risk scores (i.e. the logistic regression models were re- constructed),[1] this study essentially compares the predictive value of various combinations of clinical variables, rather than the actual risk scores per se.

Second, it is critical to recognize the inherent trade-offs between discrimination and calibration.[4] An established prognosticator with an adjusted odds ratio of 2, for example, may change the patient risk category, and yet add minimally to the c-index. Because risk scores were designed to predict different clinical outcomes at various time points, it is not feasible to directly compare their calibration---thus, comparative studies have focused on discrimination (c-statistic).[1,5,6] Nevertheless, calibration remains an important consideration in medical decision making,[4] and especially in informing patients and their families about prognosis.

Third, the electrocardiogram, troponin, and creatinine are inexpensive, routine standard investigations in ACS, and are not “difficult-to-obtain” at all. These readily accessible data can be easily incorporated into risk assessment. It should be noted that practically, even a “simple” (such as the Simple Risk Index: heart rate x [age/10]2 / systolic blood pressure) risk score (except perhaps the TIMI risk score for non-ST elevation ACS, which comprises of 7 dichotomous variables[7]) requires the use of a calculator. For example, the “linear predictor for 30-day mortality” (which will likely require a calculator) of the EMMACE model in Table 5 only provides the log odds but not the actual probability of death.[1] Therefore, the clinical utility of a risk score hinges on the ease of application at the bedside (e.g. PDA or web-based calculators are available for GRACE, Framingham, Reynolds risk scores) rather than the mere number of predictor variables.

We recognize that in some situations (e.g. in the ambulance), a very simple risk score (e.g. consisting of only a few dichotomous variables) may help to triage ACS patients more effectively. Nevertheless, in many clinical settings, a more “sophisticated” risk score can afford improved accuracy without sacrificing ease of application. Physicians should realize that these apparently “complex” risk scores can enhance risk stratification, and are in fact quite easy to use.[6,8] We contend that a useful risk score should be “as simple as possible, but not simpler”.

REFERENCES

1. Gale CP, Manda SO, Weston CF, Birkhead JS, Batin PD, Hall AS. Evaluation of risk scores for risk stratification of acute coronary syndromes in the Myocardial Infarction National Audit Project (MINAP) database. Heart 2009;95:221-227.

2. Granger CB, Goldberg RJ, Dabbous O, Pieper KS, Eagle KA, Cannon CP, Van De Werf F, Avezum A, Goodman SG, Flather MD, Fox KA. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345-2353.

3. Boersma E, Pieper KS, Steyerberg EW, Wilcox RG, Chang WC, Lee KL, Akkerhuis KM, Harrington RA, Deckers JW, Armstrong PW, Lincoff AM, Califf RM, Topol EJ, Simoons ML. Predictors of outcome in patients with acute coronary syndromes without persistent ST-segment elevation. Results from an international trial of 9461 patients. The PURSUIT Investigators. Circulation 2000;101:2557-2567.

4. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation 2007;115:928-935.

5. Yan AT, Jong P, Yan RT, Tan M, Fitchett D, Chow CM, Roe MT, Pieper KS, Langer A, Goodman SG. Clinical trial--derived risk model may not generalize to real-world patients with acute coronary syndrome. Am Heart J 2004;148:1020-1027.

6. Yan AT, Yan RT, Tan M, Casanova A, Labinaz M, Sridhar K, Fitchett DH, Langer A, Goodman SG. Risk scores for risk stratification in acute coronary syndromes: useful but simpler is not necessarily better. Eur Heart J 2007;28:1072-1078.

7. Antman EM, Cohen M, Bernink PJ, McCabe CH, Horacek T, Papuchis G, Mautner B, Corbalan R, Radley D, Braunwald E. The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. JAMA 2000;284:835-842.

8. Manfrini O, Bugiardini R. Barriers to clinical risk scores adoption. Eur Heart J 2007;28:1045-1046.