You are here

Article Text

Article menu

Download PDFPDF

Coronary artery disease
Original article
Performance of the GRACE scores in a New Zealand acute coronary syndrome cohort
  1. Aaron Lin1,
  2. Gerry Devlin2,
  3. Mildred Lee1,
  4. Andrew J Kerr1,3
  1. 1Department of Cardiology, Middlemore Hospital, Auckland, New Zealand
  2. 2Department of Cardiology, Waikato Hospital, Hamilton, New Zealand
  3. 3Section of Epidemiology and Biostatistics, University of Auckland, Auckland, New Zealand
  1. Correspondence to Dr Andrew Kerr, Department of Cardiology, Middlemore Hospital, Otahuhu, Auckland 93311, New Zealand; Andrew.Kerr{at}middlemore.co.nz

Abstract

Background Risk stratification after acute coronary syndrome (ACS) event is recommended to guide intensity and timing of investigation and management. The Global Registry of Acute Coronary Events (GRACE) investigators have published several scores for predicting patient risk both at hospital admission and discharge.

Objective To evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores for predicting myocardial infarction (MI) and mortality in a contemporary cohort of patients admitted with ACS.

Methods The cohort comprised 3743 consecutive patients admitted to cardiology services in two large New Zealand hospitals with an ACS between 2007 and 2011. Risk score data was collected in an electronic registry and linked anonymously to national hospitalisation and mortality records.

Results Between admission and 6 months, 160 patients died and another 269 were rehospitalised with an MI. The GRACE admission-to-6-month total mortality and mortality/MI scores both overestimated event rates approximately twofold. The discharge-to-6-month mortality equation was better calibrated. Global discrimination was very good for both admission-to-6-month and discharge-to-6-month mortality scores (c=0.805 and c=0.795, respectively) and moderately good for the corresponding mortality/MI equations (c=0.652 and c=0.624, respectively).

Conclusions In a contemporary ACS cohort, the GRACE discharge-to-6-month mortality score has very good discrimination and accurately predicts mortality rates, whereas the admission-to-6-month equation, despite good discrimination, overestimated risk. Recalibration or more dynamic modelling of inhospital risk which includes variables such as time from admission to risk assessment are needed to support use of ACS risk assessment inhospital.

https://doi.org/10.1136/heartjnl-2014-306062

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Early risk stratification plays a pivotal role in the management of acute coronary syndrome (ACS), as higher-risk patients are more likely to benefit from earlier and more aggressive treatment strategies.1 The Global Registry of Acute Coronary Events (GRACE) risk score is a prognostic model which encompasses the full spectrum of patients with ACS and is recommended in international guidelines.2 ,3 There were several GRACE risk scores published in the early and mid-2000s, which estimate the risk of both mortality or the composite of mortality and myocardial infarction (MI). The original Granger model predicted the risk of inhospital events only.4 The subsequent Eagle model5 estimated risk in ACS patients from discharge until 6 months postdischarge and, more recently, the Fox model6 estimates risk from admission-to-6-months after presentation.

    However, variation in the management of ACS between healthcare systems and evolving treatment strategies mean that validation of the models within the population in whom they are to be used is important. Furthermore, the GRACE equations were developed in unselected ACS admissions identified at hospital admission; but in clinical practice it is common to apply the equations in the cohort of patients admitted to a coronary care unit (CCU) with ACS. However, ACS patients who either do not survive to be admitted to CCU, or are not clinically eligible for admission to a subspecialty team, may have had a burden of comorbidity not fully captured by the GRACE equation variables. For example, malignancy, severe airways disease and sepsis are not included. If that is the case, the GRACE equations may overestimate risk in those admitted to CCUs.

    In New Zealand, all patients have a National Health Identifier (NHI) number allowing linkage of our hospital ACS registry to national hospitalisation and mortality records to identify postdischarge mortality and rehospitalisation for MI. We used this linkage between the ACS registry and national records to evaluate the performance of the admission-to-6-month and discharge-to-6-month GRACE scores in a contemporary cohort of patients admitted with ACS to two New Zealand coronary care units.

    Methods

    Consecutive patients ≥18 years old admitted to cardiology services with suspected ACS at Middlemore Hospital (catchment 500 000) between August 2007 and October 2011, and Waikato Hospital (catchment 700 000) between January 2008 and February 2010, had data recorded during their index admission by trained clinical staff using the Acute PREDICT web-based electronic database.

    Data collection

    Acute PREDICT collects admission and discharge dates, ACS risk stratification, diagnostic and inhospital investigation, management and outcome data.7 Data quality is supported by definition fields within the electronic form and 3-monthly scheduled audits of data quality.8 From 2013, this registry was rebranded as the All New Zealand ACS Quality Improvement (ANZACS-QI) registry, and is the electronic backbone of the national ANZACS-QI programme.

    Data and definitions

    Only patients with a confirmed diagnosis of non-ST-segment elevation ACS (NSTEACS, comprising both unstable angina and non-ST elevation MI (NSTEMI)) or ST-segment elevation MI (STEMI) were included. MI was defined according to the contemporary universal definition.9 The following GRACE admission-to-6-month equation variables were collected using standard definitions—patient age, admission systolic blood pressure, admission heart rate, Killip score, admission creatinine level, cardiac arrest on admission, ST segment deviation on initial ECG (ST elevation >2 mm in V1–V3 or >1 mm in other leads present in at least two contiguous leads, or ST depression ≥0.5 mm) and positive initial cardiac biomarkers. The GRACE discharge-to-6 months predictors overlap the admission-to-6-month predictors, but also include in-hospital percutaneous coronary intervention (PCI), past MI and past congestive heart failure (CHF), but not cardiac arrest. Prior MI and history of CHF were not collected in the Acute PREDICT registry. These events prior to the index admission were identified from the national hospitalisation datasets using the relevant ICD-10 codes (prior MI ICD10 I210-I214, I219-I221, I228, I229 and I252: Prior CHF I110, I130, I132, I500, I501, I509, J81).

    Outcomes

    All New Zealanders have a unique NHI number. We used an encrypted version of the NHI to anonymously link inhospital Acute PREDICT patient records to subsequent outcomes captured in national public hospitalisation and mortality datasets. The encryption and linkage methodology is described elsewhere.7 Recurrent MI, or death during the index admission, was captured in the Acute PREDICT record. Recurrent MI postdischarge was identified if there was a readmission after the index admission with a primary or secondary diagnosis of MI (ICD-10 codes I210-I214, I219-I221, I228, I229). Post-discharge deaths were identified using the national mortality data set.

    Linkage of risk to outcomes has been approved by the National Multi Region Ethics Committee (MEC/07/19/EXP).

    Statistical analysis

    To describe the baseline characteristics of the population, three risk categories were established using the cutoff points set out in the European Society of Cardiology (ESC) guidelines.2 Using the admission-to-6-months mortality equation, the three categories were: low-risk, GRACE score <88 points and the predicted mortality <3%; intermediate risk, GRACE score 88–<123 points, and the predicted mortality is 3–8%; and high-risk category, the score is ≥123 points and predicted mortality >8%. Descriptive statistics for continuous variables were summarised as means with SDs, and/or medians with interquartile ranges. Categorical data are reported as frequencies and percentages. For continuous variables, comparisons between groups were performed by the non-parametric Kruskal–Wallis test, as all data were non-normally distributed. For categorical variables, Pearson's χ2 test, or Fisher exact test were used where appropriate. All p values reported were two-tailed, and a p value <0.05 was considered significant.

    Model calibration and discrimination: For all patients, the admission-to-6-month probability of death and death/MI were calculated using the published nomograms.10 For those discharged alive, the corresponding discharge-to-6-month probabilities were calculated. The calibration of these four GRACE models was evaluated by comparing the predicted with the observed outcomes in each tertile of predicted risk. Model discrimination was then assessed by calculating the area under the receiver operating characteristic curve, otherwise known as the c-index. Additionally, the ability of the Fox admission-to-6-month mortality equation to stratify risk in patients with STEMI was compared with those with NSTEACS.

    The ability of the Fox admission-to-6-month mortality equation to predict longer-term outcomes was evaluated by estimating Kaplan-Maier survival in the ESC defined low, intermediate and high-risk subgroups. Data was analysed using SAS statistical package, V.9.2 (SAS Institute, Cary, North Carolina, USA).

    Results

    The cohort comprised consecutive New Zealand residents ≥18 years (n=3743) admitted with an ACS between 2007 and 2011 (2531 to Middlemore Hospital, 1211 to Waikato Hospital). Acute PREDICT records were completed in 99% of all patients admitted to the service. All patients had at least 6 months of potential follow-up data available through the linkage to national datasets.

    The demographics, diagnoses, risk factors, investigations and management according to risk group are shown in table 1. Europeans were more likely to be in the higher-risk category than Maori, Pacific or Indians (31% vs 18%, 20% and 16%, respectively). High-risk patients were also more likely to be older, male and have a history of cerebrovascular disease, prior MI or heart failure. High-risk patients were less likely to undergo revascularisation with PCI or coronary artery bypass surgery than lower-risk patients.

    View this table:
    Table 1

    Baseline patient characteristics according to GRACE admission to 6-month mortality score

    By 6 months after admission, 160 (4.3%) patients had died (11 (0.9%) low-risk category, 32 (2.2%) intermediate-risk category, and 117 (11.6%) high-risk category), and 429 (11.5%) patients had either died or had a MI (77 (6.0%) low-risk category, 154 (10.6%) intermediate-risk category, and 198 (19.6%) high-risk category). Figure 1 shows survival to 3 years postadmission (mean follow-up time 2.59 years) for patients in low, intermediate and high-risk groups. The curves continue to diverge beyond 6 months.

    Figure 1
    Figure 1

    Mortality to 3 years according to Global Registry of Acute Coronary Events (GRACE) admission-to-six-month mortality equation estimates.

    GRACE equation calibration and discrimination

    Calibration of the four GRACE equations was assessed by comparing the predicted and observed event rates in tertiles of risk (figure 2). Both the mortality and mortality/MI admission-to-6-month equations overestimated risk nearly twofold over all risk tertiles. The discharge-to-6-month mortality equation performed best in our cohort with only modest overestimation of risk in the highest risk tertile, and good calibration in low and intermediate tertiles. By contrast, the discharge-to-6-month mortality/MI equation markedly underestimates risk in the low and intermediate tertiles, but is well calibrated for the high-risk tertile.

    Figure 2
    Figure 2

    Calibration: Box plot with all 4 graphs with admission-to-six-month death and death/myocardial infarction (MI) (top panel) and discharge-to-six-month death and death/MI (lower panel).

    The global discrimination performance of both the admission-to-6-month and discharge-to-6-month mortality equations was similar and very good (c=0.805 (95% CI 0.770 to 0.839) and c=0.795 (95% CI 0.748 to 0.842), respectively) (figure 3). The ability of the equations to predict the composite of death/MI was more modest and similar for the admission and discharge equations (c=0.652 (95% CI 0.625 to 0.680) and c=0.624 (95% CI 0.591 to 0.657), respectively). The sensitivity and specificity of a high-risk (>8%) GRACE admission-to-6-month GRACE mortality score to predict death by 6 months were 73% and 75%, respectively.

    Figure 3
    Figure 3

    Global discrimination of the Global Registry of Acute Coronary Events (GRACE) equations.

    ACS subtypes

    The GRACE admission-to-6-month mortality score distinguished high from lower risk well for both STEMI and NSTEACS subgroups (figure 4). The GRACE score was better at stratifying lower risk NSTEACS patients than STEMI patients. The mortality rate was higher in intermdiate compared with the low-risk patients with NSTEACS (p=0.003) but not those with STEMI (p=1.000). The overall discrmination was similar for both subtypes (c–index for STEMI=0.757 (95% CI 0.671 to 0.844) and NSTEACS=0.820 (95% CI 0.783 to 0.858) (p=0.189)).

    Figure 4
    Figure 4

    Six-month mortality according to ESC recommended Global Registry of Acute Coronary Events (GRACE) risk categories for ST-segment elevation MI (STEMI) and non-ST-segment elevation ACS (NSTEACS). *Shows p values from the comparison between the 3 risk categories within each group (for STEMI low, n=172; intermediate, n=324; high, n=307. For NSTEACS low, n=1111; intermediate, n=1128; high, n=701).

    Discussion

    In this contemporary New Zealand ACS cohort, the GRACE mortality equations performed well at discriminating between risk strata both at 6 months and up to 3 years after discharge. They also discriminated well across both STEMI and NSTEACS subgroups, but had more modest ability to stratify risk of the composite of death/recurrent MI. However, calibration—the ability of the GRACE scores to estimate the event rate—was more modest. The best calibrated equation was the ‘Eagle’ discharge-to-6-month mortality equation. By contrast, there was a nearly twofold overestimation of risk using the Fox admission-to-6-month equations in our cohort.

    Discrimination compared with prior validation studies

    The performance of the four equations in stratifying risk was only slightly inferior to that described in the original GRACE validation cohorts.5 The c statistic for the Fox admission-to-6-month mortality and death/MI equations in their validation cohort were 0.82 and 0.73, respectively, compared with 0.81 and 0.65 in our cohort.6

    Calibration of the admission-to-six month equations

    The variability in calibration between the equations and between our cohort and the validation cohorts may, in part, be due to changes in clinical practice which are not accounted for by variables in the admission-to-6-month calculator. This is explored in table 2 where we have shown the medical history, diagnoses, GRACE score risk factors and discharge medication reported in the original GRACE papers and corresponding data for our cohort. The two most notable differences are the higher rates of revascularisation and of secondary prevention medication use in our cohort. Both these variables would be expected to reduce subsequent adverse events, but neither are included in the admission-to-6-month calculator.

    View this table:
    Table 2

    Comparison between the GRACE derivation and ANZACS-QI cohorts

    A further consideration relevant to calibration is the ACS cohort definition. The GRACE cohort included patients admitted to hospital with suspected ACS who also had ECG changes, elevated cardiac markers or known coronary artery disease. Previous GRACE validation studies have varied in the comprehensiveness of their ACS capture ranging from undifferentiated chest pain admissions,11 confirmed discharge diagnosis of ACS,12 CCU admission with ACS,13 ,14 to ICD10 codes for ACS.15 We included all diagnoses of ACS including those without ECG or cardiac biomarker changes, and so may have had a slightly lower-risk cohort, although if variable selection and weighting is appropriate this should not affect calibration performance. Probably more importantly, our cohort included only those who survived to be admitted to the cardiology service. Those dying in the emergency department or intensive care unit were not captured. It is likely that these patients who were included in the derivation cohorts have a burden of comorbidity not captured in the equations, for example, those with very severe underlying cardiac disease, malignancy, severe airways disease and sepsis. If, as is likely, these uncaptured risk factors are over-represented in the patients who died prior to CCU admission, a lower-risk cohort remains to be admitted to CCU, and the GRACE equations would correspondingly overestimate risk in those patients. This may account for some of the variation in reported calibration performance of the equations.15 ,16 This changing risk as a function of length of hospital stay has been previously investigated in STEMI cohorts with more dynamic modelling of risk which adjusts for variables, such as time from admission to risk assessment,17–19 and degree of ST segment resolution at 90 min after thrombolysis.20 We are unaware of any similar dynamic modelling of risk in NSTEACS.

    Performance of the discharge-to-6-month equations

    Both the Eagle discharge-to-6-month cohort and ours comprised a similar cohort—ACS survivors at hospital discharge. It is therefore reassuring that the observed and predicted mortality rates were similar. What is perhaps more surprising is that event rates in the lowest risk tertiles in our cohort were higher than predicted by the Eagle discharge-to-6-month death/MI equation. One reason for this may be the adoption of lower threshold for diagnosis of MI since the 2003 adoption of the new universal definition of MI.9 Readmissions which were classified as unstable angina are now more likely to be labelled MI, which may contribute to the observed higher than predicted rates. Differences between the cohorts in risk factors not included as variables in the equations may also contribute to the observed differences. For example the rate of diabetes in our cohort is higher than in the GRACE validation cohorts.

    Prediction of composite outcomes

    The GRACE equations were developed to stratify risk following an ACS presentation to guide early treatment decisions. They do well in stratifying risk of death after ACS, however, performance is more modest in predicting the composite death/MI outcome. This suggests that prediction of recurrent MI using these equations is poor. Several potential risk factors for recurrent MI were not included in the final GRACE model because these variables, collected at admission, were not significant independent predictors. These included presence of diabetes, smoking status and lipid levels. However, the lack of predictive value of these admission values may be because there are often significant changes in these variables as a result of inhospital diagnosis of diabetes, lifestyle change and medication initiated at the time of the ACS event. In the future it may be more useful to develop separate equations based on risk factor data obtained in a stable cohort late after the ACS event.

    Prior external validation

    There are no prior published validation studies for the Fox admission-to-6 months equation using an external cohort. There are several for the Eagle discharge-to-6-month mortality equation,12 ,13 ,15 ,21–23 but none for the death/MI equation. Discrimination using this equation was very good in all external cohorts. Calibration was reported in only four of these studies, with good calibration in three,12 ,21 ,22 and underestimation of NSTEMI risk in another.15

    In our study, we did not assess performance of the Granger inhospital mortality equation or the more recent updated inhospital model24 due to a small number of inhospital deaths. Discrimination performance of the Granger equation has been proven in a number of external validation cohorts,11 ,12 ,21 ,23 ,25 although only three12 ,21 ,25 reported calibration performance.

    Clinical implications

    Both accurate discrimination and calibration are needed from a risk prediction equation. Clinicians need to know which ACS patients are at highest risk so they can be prioritised for earlier and more aggressive treatment. But they also need to know what the absolute risk is, so patients can be advised regarding the risk and benefits of various treatment options, and to support them in making rational cost-benefit decisions. From this study, the GRACE discharge-to-6-month mortality equation provides the most reliable information for all these purposes at the time of discharge. By contrast, using the admission-to-6-month calculator in this cohort markedly overestimates event rates and, therefore, any estimated benefits of treatment. By the time of hospital discharge many of the most important decisions regarding pharmacotherapy and revascularisation have already been made. The discharge-to-6-month equation may, nevertheless, be useful in identifying patients for more intensive cardiac rehabilitation and medication adherence monitoring.

    A prior GRACE validation study found that the risk of inhospital mortality varied over time for some regions of the world.24 Taken together with our results this supports the importance of regular updating of these risk models within the populations for whom they are used.

    Where to next?

    In 2013, the NZ Ministry of Health in conjunction with the NZ Cardiac Clinical Network commenced funding of a national ACS and cardiac procedures registry and quality improvement programme known as the ANZACS-QI. In New Zealand, routine capture of ACS data in the ANZACS-QI registry, and regular linkage of data subsequent to national outcome datasets will allow us to develop and regularly update NZ-specific ACS scores.

    Conclusion

    When applied in a contemporary CCU cohort, the GRACE discharge-to-6-month mortality equation has very good discrimination and accurately predicts mortality rates, whereas, the admission-to-6-month equation, despite good discrimination, overestimates risk. Recalibration, or more dynamic modelling of inhospital risk which includes variables such as time from admission-to-risk assessment, are needed to support use of ACS risk assessment inhospital.

    Key messages

    What is already known on this subject?

    • The Global Registry of Acute Coronary Events (GRACE) risk score is a prognostic model which encompasses the full spectrum of patients with acute coronary syndromes (ACS). There were several GRACE risk scores published in the early and mid-2000s which estimate the risk of both mortality and the composite of mortality and myocardial infarction (MI).

    What might this study add?

    • In this contemporary New Zealand cohort of ACS patients admitted to cardiology services, the GRACE mortality equations performed well at stratifying risk both at 6 months and up to 3 years after discharge. They had a more modest ability to stratify risk of the composite outcome of death/MI. The best calibrated equation was the Eagle discharge-to-6-month mortality equation. By contrast, there was a nearly twofold overestimation of risk using the Fox admission-to-6-month equations.

    How might this impact on clinical practice?

    • Both accurate discrimination and calibration are needed from a risk prediction equation. Clinicians need to know which ACS patients are at highest risk so they can be prioritised for earlier and more aggressive treatment. But they also need to know what the absolute risk is, so patients can be advised regarding the risk and benefits of various treatment options and make rational cost-benefit decisions. From this study, the GRACE discharge-to-6-month mortality equation provides the most reliable information for all these purposes at the time of discharge. By contrast, using the admission-to-6-month calculator in this cohort overestimates event rates and, therefore, any extrapolated benefits of treatment.

    Acknowledgments

    Our thanks to the National Health Board Analytic Services and Pharmac for enabling use of this data. We would like to thank Professor Rod Jackson for his critical review of the manuscript.

    References

    1. ↵
      1. Mehta SR,
      2. Granger CB,
      3. Boden WE,
      4. et al
      . Early versus delayed invasive intervention in acute coronary syndromes. N Engl J Med 2009;360:2165–75.
      OpenUrlCrossRefPubMedWeb of Science
    2. ↵
      1. Hamm CW,
      2. Bassand J-P,
      3. Agewall S,
      4. et al
      . ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: The Task Force for the management of acute coronary syndromes (ACS) in patients presenting without persistent ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J 2011;32:2999–3054.
      OpenUrlFREE Full Text
    3. ↵
      Non ST-Elevation Acute Coronary Syndrome Guidelines Group and the New Zealand Branch of the Cardiac Society of Australia and New Zealand. New Zealand 2012 guidelines for the management of non ST-elevation acute coronary syndromes. N Z Med J 2012;125:122–47.
      OpenUrlPubMed
    4. ↵
      1. Granger CB,
      2. Goldberg RJ,
      3. Dabbous O,
      4. et al
      . Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345–53.
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      1. Eagle KA,
      2. Lim MJ,
      3. Dabbous OH,
      4. et al
      . A validated prediction model for all forms of acute coronary syndrome: estimating the risk of 6-month postdischarge death in an international registry.[see comment]. JAMA 2004;291:2727–33.
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Fox KAA,
      2. Dabbous OH,
      3. Goldberg RJ,
      4. et al
      . Prediction of risk of death and myocardial infarction in the six months after presentation with acute coronary syndrome: prospective multinational observational study (GRACE). BMJ 2006;333:1091.
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Kerr AJ,
      2. Looi JL,
      3. Garofalo D
      et al. Acute Predict: a clinician-led cardiovascular disease quality improvement project (Predict-CVD 12). Heart Lung Circ 2010;19:378–83.
      OpenUrlCrossRefPubMed
    8. ↵
      1. Garofalo D,
      2. Grey C,
      3. Lee M
      et al. Pre-hospital delay in acute coronary syndromes: PREDICT CVD-18. N Z Med J 2012;125:12–22.
      OpenUrlPubMed
    9. ↵
      1. Thygesen K,
      2. Alpert JS,
      3. White HD
      et al. Universal definition of myocardial infarction. Eur Heart J 2007;28:2525–38.
      OpenUrlFREE Full Text
    10. ↵
      1. Anderson F,
      2. Fitzgerald G
      . Methods and formulas used to calculate the GRACE Risk Scores for patients presenting to hospital with acute coronary syndrome. 2010, Copyright 1998–2010, Center for Outcome Research, University of Massachusetts Medical School.
    11. ↵
      1. Ramsay G,
      2. Podogrodzka M,
      3. McClure C,
      4. et al
      . Risk prediction in patients presenting with suspected cardiac pain: the GRACE and TIMI risk scores versus clinical evaluation. QJM 2007;100:11–18.
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Aragam KG,
      2. Tamhane UU,
      3. Kline-Rogers E,
      4. et al
      . Does simplicity compromise accuracy in ACS risk prediction? A retrospective analysis of the TIMI and GRACE risk scores. PLoS ONE 2009;4:e7947.
      OpenUrlCrossRefPubMed
    13. ↵
      1. Tang EW,
      2. Wong C-K,
      3. Herbison P
      . Global Registry of Acute Coronary Events (GRACE) hospital discharge risk score accurately predicts long-term mortality post acute coronary syndrome.[Erratum appears in Am Heart J. 2007 Nov;154(5):851]. Am Heart J 2007;153:29–35.
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. de Araujo Goncalves P,
      2. Ferreira J,
      3. Aguiar C
      et al. TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005;26:865–72.
      OpenUrlAbstract/FREE Full Text
    15. ↵
      1. Bradshaw PJ,
      2. Ko DT,
      3. Newman AM
      et al. Validity of the GRACE (Global Registry of Acute Coronary Events) acute coronary syndrome prediction model for six month post-discharge death in an independent data set. Heart 2006;92:905–9.
      OpenUrlAbstract/FREE Full Text
    16. ↵
      1. Elbarouni B,
      2. Goodman SG,
      3. Yan RT
      et al. Validation of the Global Registry of Acute Coronary Event (GRACE) risk score for in-hospital mortality in patients with acute coronary syndrome in Canada. Am Heart J 2009;158:392–9.
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Chang W-C,
      2. Boersma E,
      3. Granger CB
      et al. Dynamic prognostication in non-ST-elevation acute coronary syndromes: insights from GUSTO-IIb and PURSUIT. Am Heart J 2004;148:62–71.
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Chang W-C,
      2. Kaul P,
      3. Fu Y
      et al. Forecasting mortality: dynamic assessment of risk in ST-segment elevation acute myocardial infarction. Eur Heart J 2006;27:419–26.
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Westerhout CM,
      2. Pieper KS,
      3. James SK
      et al. Dynamic modeling of 90-day mortality in ST-elevation myocardial infarction patients undergoing primary percutaneous coronary intervention. Am Heart J 2013;165:354–62.e2.
      OpenUrlCrossRefPubMed
    20. ↵
      1. Sherwood MW,
      2. Morrow DA,
      3. Scirica BM
      et al. Early dynamic risk stratification with baseline troponin levels and 90-minute ST-segment resolution to predict 30-day cardiovascular mortality in ST-segment elevation myocardial infarction: Analysis from CLopidogrel as Adjunctive ReperfusIon TherapY (CLARITY)—Thrombolysis in Myocardial Infarction (TIMI) 28. Am Heart J 2010;159:964–971.e1.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Abu-Assi E,
      2. Ferreira-Gonzalez I,
      3. Ribera A,
      4. et al
      . “Do GRACE (Global Registry of Acute Coronary events) risk scores still maintain their performance for predicting mortality in the era of contemporary management of acute coronary syndromes?” Am Heart J 2010;160:826–34.e1-3.
      OpenUrlCrossRefPubMed
    22. ↵
      1. Alter DA,
      2. Venkatesh V,
      3. Chong A,
      4. et al
      . Evaluating the performance of the Global Registry of Acute Coronary Events risk-adjustment index across socioeconomic strata among patients discharged from the hospital after acute myocardial infarction. Am Heart J 2006;151:323–31.
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Gale CP,
      2. Manda SOM,
      3. Weston CF
      et al. Evaluation of risk scores for risk stratification of acute coronary syndromes in the Myocardial Infarction National Audit Project (MINAP) database. Heart 2009;95:221–7.
      OpenUrlAbstract/FREE Full Text
    24. ↵
      1. Pieper KS,
      2. Gore JM,
      3. FitzGerald G
      et al. Validity of a risk-prediction tool for hospital mortality: the Global Registry of Acute Coronary Events. Am Heart J 2009;157:1097–105.
      OpenUrlCrossRefPubMedWeb of Science
    25. ↵
      1. Schiele F,
      2. Meneveau N,
      3. Seronde MF,
      4. et al
      . C-reactive protein improves risk prediction in patients with acute coronary syndromes. Eur Heart J 2010;31:290–7.
      OpenUrlAbstract/FREE Full Text

    Footnotes

    • Contributors AL, GD and AJK, were involved in the design of the study. ML analysed the data. AL, GD, ML and AJK interpreted the data. AJK and AL drafted the manuscript, and all authors are responsible for critically reviewing it and approving the final submission.

    • Funding This research project has been supported by the Health Research Council (grant number 11/800).

    • Competing interests AL was supported by the Middlemore Hospital Cardiac Trust.

    • Ethics approval New Zealand Multi-region ethics Committee.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement Request for access to this data can be made to the Chair of the ANZACS-QI governance group via the corresponding author where they will be considered according to the established governance protocol.