Objective To compare the accuracy of the GRACE score, a strong prognosticator in acute coronary syndrome (ACS) that is calculated using conventional cardiac troponin (cTn) assays, with that calculated with high-sensitivity cTn (hs-cTn) and with the combination of the GRACE score with hs-cTn or B-type natriuretic peptide (BNP).
Design Prospective international cohort.
Settings University Hospital.
Patients Patients enrolled in the Predictors of Acute Coronary Syndromes Evaluation prospective study with proven ACS.
Main outcome measured The capacity to predict in-hospital mortality, 1-year mortality and combined death/acute myocardial infarction (AMI) at 1 year.
Results 370 patients were enrolled (173 with unstable angina and 197 with AMI). In-hospital mortality was 4.1%; 1-year mortality was 12.5%. The GRACE score was significantly higher in patients who died than in those discharged alive (200 (174–222) vs 125 (98–155); p<0.001), and in those who died than in those who survived for 1 year (151 (133–169) vs 104 (85–125); p<0.001). The area under the curve of the GRACE score was 0.87 regarding in-hospital mortality and 0.88 for 1-year mortality; if calculated with hs-cTn, it was 0.87 and 0.88, respectively (p=NS for all comparisons). The addition of hs-cTn to the GRACE score resulted in no increased value, whereas the addition of BNP tended to improve 1-year mortality prediction (p=0.058).
Conclusion The GRACE score is accurate for determining both in-hospital and long-term mortality in patients with ACS in the era of hs-cTn. The addition of hs-cTn or BNP to the GRACE score does not significantly improve risk prediction.
- Acute coronary syndrome
- GRACE score
- cardiac troponins
- B-type natriuretic peptide
- risk stratification
Statistics from Altmetric.com
- Acute coronary syndrome
- GRACE score
- cardiac troponins
- B-type natriuretic peptide
- risk stratification
Acute coronary syndrome (ACS) is a major health problem.1 2 As considerable variability exists among patients across the spectrum of ACS, accurate determination of risk has become a major focus in their initial evaluation.1–4 Numerous risk-prediction models for different types of patients with ACS exist, but the most widespread tool is the GRACE risk score.3–8 The GRACE score includes basic data from a large unselected population of patients with ACS, including those with ST-elevation acute myocardial infarction (STEMI), non-STEMI (NSTEMI) and unstable angina (UA).4 Initially designed to identify high-risk patients for in-hospital mortality, 4 a second, simplified, model has been developed to predict 6-month mortality.5 The excellent ability of GRACE scores to discriminate the risk of death or combined death and acute myocardial infarction (AMI) has been confirmed for both 6 months 9 and more prolonged periods up to 6 years.6 10–12 Its use has recently been recommended by the European Society of Cardiology for risk stratification in patients with non-ST-elevation ACS.2
Several biomarkers not considered in the previous scores have emerged as potential prognosticators in ACS, pointing to the need to re-evaluate scoring systems in the light of these new risk predictors. The recently developed high-sensitivity cardiac troponin (cTn) assays (hs-cTn) have greater accuracy than conventional assays in the early diagnosis of AMI.13 14 However, concerns have been raised about possible increased hs-cTn levels in patients with a final diagnosis other than AMI and in low-risk patients.15 16 Whether its incorporation into the calculation of the GRACE score will alter or improve the model and the effect of its addition to the GRACE score are not known.
The B-type natriuretic peptides (BNP and NT-proBNP) are strong prognosticators in ACS, independently of cTn.17 18 The possible merit of their combination with the GRACE score varies among studies and their real effect remains unknown.11 19 20 To resolve these issues, we compared the risk prediction conferred by the GRACE score using conventional cTn assay with hs-cTn assay for its calculation, and the combination of the GRACE score with hs-cTn or BNP measurements.
The study population was derived from the Advantageous Predictors of Acute Coronary Syndromes Evaluation study, the first results of which have been reported.13 Briefly, the Advantageous Predictors of Acute Coronary Syndromes Evaluation is an ongoing prospective, international, multicentre study, designed and coordinated by the University Hospital Basel, which enrolled all patients with suspected AMI of less than 12 h.
The present study analysed the subgroup of patients with a final confirmed diagnosis of ACS, covering its full spectrum, from UA to NSTEMI and STEMI. The final diagnosis of ACS was based on all clinical, laboratory (but not hs-cTn and BNP concentrations) and imaging data available and was separately confirmed by two cardiologists. Disagreements between the observers were settled by a third cardiologist. AMI was defined according to the universal definition and required the association of clinical signs of myocardial ischaemia with the observation of a rising or falling pattern of local cTn concentration; when the 99th centile of the test could not be measured with the precision required, the 10% coefficient variation (CV) level was used as the cut-off value.21 22 UA was diagnosed in the presence of (1) clinical manifestations suggesting myocardial ischaemia, without evidence of myocardial necrosis and (2) an ECG indicating ongoing ischaemia, or a >70% stenosis of an epicardial coronary artery on coronary angiography, or a positive result on a cardiac exercise test, or when the diagnosis was uncertain but follow-up information showed that the patient suffered an AMI or sudden unexpected cardiac death within 60 days.
The ethics committee at each institution approved this study, and all patients gave written informed consent to participate.
Measurement of biomarkers
Blood samples were collected in EDTA tubes at admission and centrifuged at 4°C within 15 min of collection. Conventional cTn assays were immediately analysed, and the results were available to the emergency department practitioner. The following cTn assays were used for the clinical care of the patients at the participating hospitals: Abbott Axsym cTnI ADV (limit of detection 0.02 μg/l; 99th centile cut-off point <0.08 μg/l; CV ≤10% for 0.16 μg/l), Beckmann Coulter Accu cTnI (limit of detection 0.01 μg/l; 99th centile cut-off point <0.04 μg/l; CV ≤10% for 0.06 μg/l) and Roche cTn 4th generation (limit of detection of 0.01 μg/l; 99th centile cut-off point <0.01 μg/l; CV ≤10% for 0.035 μg/l). Plasma hs-cTn concentrations were measured using a commercially available electrochemiluminescence immunoassay (hs-cTn, Roche Diagnostics, Mannheim, Germany). The 99th centile with a CV ≤10% is achieved for 14 ng/l.23 BNP was measured using the commercially available Biosite Diagnostics assay (Biosite Diagnostics, La Jolla, California, USA); its analytical sensitivity is <5.0 pg/ml, with a measurable range of 0–5000 pg/ml.
Calculation of GRACE score
For in-hospital prediction, the GRACE score was calculated on the basis of admission characteristics using the followings items: Killip class, systolic blood pressure, heart rate, age, creatinine concentration, cardiac arrest, the presence of any ST-segment deviation, the presence/absence of elevated cardiac markers.4
For long-term prediction, the simplified GRACE model was calculated on the basis of admission characteristics using the following items: age, previous heart failure, past AMI, heart rate, systolic blood pressure, the presence of ST-segment depression, creatinine concentration, elevated cardiac markers, no in-hospital percutaneous coronary intervention.5
For the present analysis, a single value from the conventional assay of cTn above the cut-off value was determined as a score for elevated biomarkers of myocardial necrosis.
Studied end points
We measured the capacity of the GRACE score to predict in-hospital death, 1-year mortality and combined death/AMI at 1 year. We chose to evaluate the end points at these two time points because they represent short- and long-term follow-up periods, respectively. In addition, the prognostic significance of both the GRACE score and BNP has been demonstrated at 1 year.12 17
The data are expressed as means±SD for Gaussian continuous variables, median (IQR) for non-Gaussian continuous variables, and numbers and percentages for categorical variables. The baseline characteristics were analysed using the χ2 test or Fisher exact test and the Student t test or Mann–Whitney test, as appropriate, for comparison between groups defined by the presence or absence of the end points. The existence of a correlation between BNP, hs-cTn and the GRACE risk score was investigated using the Spearman correlation test. The possible role played by variables in relation to the end points was examined first in a single-variable analysis, followed by logistic regression. We then compared the predictive accuracy of the GRACE score with its recalculation with hs-cTn, the combined GRACE+hs-cTn, GRACE+BNP using receiver operating characteristics curves, and integrated discrimination improvement (IDI) (figure 1).24 25 p<0.05 was considered significant. Stata statistical software V.10.1 was used for all analyses.
The study population consisted of 370 patients comprising 50 with STEMI, 147 with non-STEMI, and 173 with UA. Using hs-cTn rather than the conventional assay of cTn, in conjunction with the universal definition for the confirmed diagnosis of AMI, would have led to reclassification of 28 patients from UA to AMI.21 The GRACE score shifted from 120 (102–148) to 134 (109–156) in these patients (p<0.267). No patients were lost to follow-up. Follow-up was 100% complete at discharge from hospital and 84.3% complete at 1 year. The detailed characteristics of the study group, including comparison of the survivors and non-survivors, are shown in table 1.
The GRACE score for predicting in-hospital outcome was 126 (99–159) in the entire cohort, ranging from149 (123–185) in patients with STEMI to 123 (96–155) in patients with UA/NSTEMI. Fifteen patients (4.1%) died during hospitalisation. Compared with the patients who were discharged alive, the patients who died during initial hospitalisation were older and had a higher heart rate and reduced glomerular filtration rate at admission (table 1). In addition, they had increased hs-cTn (78 (28–224) vs 27 (10–119) ng/l, respectively; p=0.015) and BNP (487 (229–885) vs 135 (64–338) pg/ml; p=0.005).
There was a significant but only partial correlation between the GRACE risk score and hs-cTn concentrations (r=0.54, p<0.001) or BNP (r=0.62, p<0.001). The GRACE score was increased in patients who died versus those discharged alive (200 (174–222) vs 125 (98–155); p<0.001). The GRACE score is the only variable that remains associated with in-hospital mortality according to multivariate analysis (p<0.001). Table 2 presents the predictive accuracy of the different models. The consideration of increased hs-cTn (above the 99th centile) rather than increased conventional assay of cTn for the calculation of the GRACE score did not alter the model. The addition of neither hs-cTn concentration nor BNP concentration to the GRACE score improved the model (table 2).
The patients mostly had a low (score <90, 27.5%) or intermediate (score from 90 to 120, 35.8%) risk of 1-year death as assessed by the GRACE score. Thirty-nine (12.5%) patients died during the 1-year follow-up period, and 56 reached the combined end point of death/recurrence of AMI (18.0%). The variables that are associated with 1-year mortality are reported in table 1. Patients who died had higher hs-cTn concentrations (147 (30–470) vs 23 (9–103) ng/l; p<0.001), BNP concentrations (558 (322–1183) vs 120 (58–249) pg/ml; p<0.001) and GRACE score (151 (133–169) vs 104 (85–125); p<0.001) than those who survived. The GRACE risk score (p<0.001) and the presence of ST-segment deviation (p=0.002), in-hospital revascularisation (p=0.011) and glomerular filtration rate (p=0.022) remained associated with 1-year mortality in multivariate analysis. The GRACE score accurately predicted the 1-year mortality end point. Its calculation using increased hs-cTn did not alter the model (table 2). The addition of hs-cTn to the GRACE score did not improve the model. There was a trend for an improvement conferred by combining BNP concentrations with the score, as expressed by the IDI but not c-statistics (table 2).
Lastly, the GRACE risk score predicts correctly, but less adequately, the composite end point of death/recurrence of AMI at 1 year. The model was neither altered by the consideration of increased hs-cTn nor improved by the addition of hs-cTn or BNP consideration to the risk score (table 2).
We made clinically important observations in this study. First, the GRACE risk score accurately predicted both in-hospital and 1-year mortality, and less accurately the combined 1-year death and AMI. Second, the use of increased hs-cTn instead of conventional cTn in the calculation of the GRACE risk score did not alter the model. Third, the combination of hs-cTn or BNP concentrations with the GRACE score had no significant incremental value.
The in-hospital and 1-year prognostic values of the GRACE score were excellent in our study, and we observed even higher AUC than in the validation sets of the GRACE registries.4 5 9 Such findings were expected and emphasise that a prediction tool is as generalisable as the population from which it is derived. The GRACE registry is a multinational registry involving 94 hospitals in 14 countries; it includes more than 10 000 patients and was designed for prognosis of all-cause mortality. Its accuracy in the prediction of mortality has been confirmed in several studies at different time points.6 10 12 In addition, the GRACE score predicted fairly, but less accurately, combined 1-year death/AMI, a result that is consistent with previous studies.9 20 More importantly, we observe that the use of increased hs-cTn rather than conventional cTn for its calculation does not alter the model.
Over the last few decades, cTn has emerged as the key biomarker for detecting ACS in patients with chest pain-related symptoms.26 27 Recently, high-sensitivity assays of cTn have demonstrated their superiority over conventional assays for detecting AMI and overall ACS.13 14 28 They are scheduled to soon become the only available cTn assays. Although some authors have reported a possible specificity deficit of these new assays,15 16 our study demonstrates that the GRACE risk calculation is still valid with the use of these new assays. This may have important clinical implications. Initial risk stratification of patients with ACS is crucial for making appropriate decisions about the need for transfer to a tertiary centre. In addition, patients at highest risk may derive greater benefits from intense pharmacological treatment and interventions, the benefits of which may outweigh the risk of adverse effects.1 2
BNP is a marker of cardiac stress and its accuracy as a strong prognosticator has been proven even in the absence of left ventricular dysfunction and independently of certain elements of the GRACE score such as age, ST deviation and cTn concentration.17 18 29 However, there are conflicting results on whether the addition of BNP concentration to the GRACE score improves risk stratification. Beygui and colleagues demonstrated in a retrospective analysis of the ARCHIPELAGO study (irbesartan in patients with ACS without ST segment elevation) that the performance of the GRACE score was significantly improved by the introduction of BNP concentration to predict death/heart failure but not the composite end point of death, stroke, AMI, recurrent ischaemia or unplanned hospitalisation.20 In addition, this study investigated patients with long-lasting symptoms (up to 48 h) at low risk of cardiac events (2-month mortality of 0.7%). In another study, Ang and colleagues reported that BNP concentration did not improve the risk prediction offered by the GRACE score, as expressed by the c-statistics.19 Our results are consistent with these studies, as we report no significant difference in the c-statistics or the IDI offered by the addition of BNP concentrations to the GRACE score.
In this study, we examined patients across the entire spectrum of ACS and focused on mortality because the GRACE score was developed and validated mainly for this indication. We demonstrate that neither BNP nor hs-cTn added significant value to the GRACE score. Subgroup analysis showed no difference in in-hospital mortality prediction (p=0.497 and p=0.930, respectively) and 1-year mortality (p=0.802 and p=0.860, respectively) of the different models in patients with STEMI and UA/NSTEMI, respectively. The possible merit of using inflammatory markers, structural enzymes, markers of platelet activation and neurohormones in addition to the GRACE score has been previously investigated. Only interleukin-6 demonstrated incremental value.11 19 20 These previous results and ours should not be considered to be a contradiction of the prognostic significance of either hs-cTn or BNP. As we report a very high AUC (>0.85) in all our analyses, our results should rather be interpreted as the high capacity to discriminate patients at high risk offered by the GRACE score.
Receiver operating characteristics analysis is designed ideally to measure the accuracy of a single predictor across its broad concentration. Such analysis can also easily compare two single predictors in a head-to-head fashion, but is not ideal for comparing a combination of predictors with a single predictor. In addition, it may lack sensitivity. Recently, the net reclassification improvement and IDI have emerged as the preferred methods for investigating the predictive ability of new biomarkers or new risk scores.25 In the present study, we compared the predictive ability of the different models using both c-statistics and the IDI; we assume therefore that our results are robust.
Our study should be interpreted within its limitations. First, patients with severe renal failure were excluded from the study. Therefore, no conclusion can be drawn about this specific population. Second, the sample size is relatively small compared with the registries that were used to create and validate the GRACE score.4 5 However, we found a similar accuracy of the GRACE score to that reported in these studies, which may suggest that our population is an adequate representation of the spectrum of patients with ACS. In our study, the rate of in-hospital percutaneous coronary intervention ranged from 45% for UA to 80% for STEMI. As cardiac markers are related to the extent of coronary disease and possible revascularisation, cTn and hs-cTn may have lost their prognostic effect after successful revascularisation. Other components of the GRACE score maintain prognostic relevance, explaining the absence of difference that we observed in the prognostic significance of the GRACE score using different cardiac markers for its calculation.
The GRACE risk prediction score is still accurate for determining both in-hospital and long-term mortality in patients with ACS in the era of high-sensitivity assays of cardiac troponin. The addition of hs-cTn or BNP to the GRACE score does not improve the risk prediction offered by the GRACE score alone.
See Editorial, p 1461
Funding This study was supported by grants from the Swiss National Science Foundation (PP00B-102853), the Swiss Heart Foundation, Abbott, Roche, Siemens and the Department of Internal Medicine, University Hospital Basel. CM was supported by a grant from the Freie Akademische Gesellschaft Basel (FAG).
Competing interests CM received research grant support from Abbott, Brahms, nanosphere, Roche and Siemens, consulting fees from Abbott, and lecture fees from Abbott, Biosite, Brahms, Roche and Siemens.
Patient consent Obtained.
Ethics approval This study was conducted with the approval of the University Hospital Basel.
Provenance and peer review Not commissioned; externally peer reviewed.