Send letter to journal:
Re: ACS Scores: Risky business?

To the Editor:

If a study identifies several predictors of high risk of adverse events - as GRACE has attempted to do for ACS, could we dare presume that the absence of those predictors might perhaps indicate low risk i.e. freedom from those events? Have we somehow missed the point of this paper by Brieger et al? Or are there more analyses just waiting to be presented from a different angle to further advance our ability to manage this multifaceted condition called ACS?

Send letter to journal:
Re: this article

To the editor. We read with interest the paper presented by Breiger et al [1] and feel that it is most worthy of comment. The authors describe the development and internal validation of a risk score for patients with NSTEACS, from a global registry, that predicts in-patient freedom-from- events. They present an alternative concept to ACS risk characterisation – rather than the identification of high-risk subjects who merit more intensive management, they elect to create a model that is able to assist healthcare workers identify lower risk patients who may not benefit as much from such therapies. They argue that this is important because the discriminative performance of ACS risk scores designed to identify patients at higher risk of adverse outcomes is limited when applied to patients at lower risk of adverse events. We believe that several key issues deserve clarification.

1) Whereas some ‘near-point’ ACS risk models are often based on a combination of a retrospective final diagnosis and data gathered over a period of some time (‘far-point’ data), the model proposed by Breiger et al is generated from a NSTEACS cohort using variables that may be collected within 24 hours of admission. Even so, the authors appear to have retrospectively identified the NSTEACS cohort from the unselected GRACE ACS population and wonder why the model was not generated from the unadulterated population.

2) The authors have quite correctly noted that their Freedom-from- Event score is not as accurate a predictor as the 15 individual patient characteristics. This may be due to complex interactions between these characteristics: for example the effect of higher Killip class may be different in a 50 year-old male than in an 80 year-old female.

3) The GRACE registry has revealed substantial differences in the management of patients based on hospital type and geographical location [2]. Indeed, one could envisage different hospitals or countries having different thresholds for NSTEACS management. The hierarchical structure of the data (that is patients within hospitals within countries) might therefore be considered during model development because it could influence the coefficients of predictor variables, their precision, and consequently the covariates that might be considered important to include in the model. Do the authors have evidence to demonstrate the performance of their score at the regional level? Options might include tuning the modelling to a particular country or using data from the country in which the model will be used in. Data from MINAP may allow the development of models that may be more applicable to the English and Welsh population.

4) The impact of prior use of medical therapies is noteworthy: statins (for which there is exhaustive RCT evidence supporting their benefit) show only a marginal benefit and warfarin 'appears' marginally harmful. It is difficult to interpret these appearances in relation to the true benefit of the therapies. The use of warfarin for example could be a proxy indicating that a patient is at high risk perhaps because of atrial fibrillation. The use of statins may reflect a higher risk population by virtue of its prescription based on the patient’s past history. There is no doubt that prior medical therapies should be included in the model for risk but, as the authors are careful to avoid, their coefficients in the model should not be interpreted without caution. We note that aspirin does not appear in the final model. Understandably, there is a balance between the inherent risks in patients who are already taking aspirin and its derived benefits, which may in this model, cancel out. The inferences described are derived from an observational database and not a RCT and interpretation as a consequence is not straightforward.

5) We wonder how the authors overcome the potential overestimation of outcomes in patients presenting with features that could be considered an outcome variable. That is, was atrial fibrillation as an outcome measure excluded in patients admitted with chronic atrial fibrillation?

6) The authors do not appear to consider the impact of in-patient therapies. We wonder to what degree, for example, lack of emergency aspirin therapy influences outcome for NSTEMI patients.

7) We recognise that GRACE is an extensive ACS registry with in- patient mortality status missing for only 2% of the cohort [3]. The authors, however, do not state the extent of data missingness for other output variables such as bleeding, or covariates such as age or systolic blood pressure. Earlier work arising from the registry suggested that imputation methods do not affect the main result [3], implying that an assumption of ‘missing at random’ might be made. Indeed this is an assumption that modellers of such observational data would consider reasonable. With such a range of countries and sites within them however, it would be interesting to have the authors comments on the potential for differential data missingness (greater missingness in some countries or hospitals) to influence estimates in their model?

8) We note the mapping from Freedom-from-Event score to outcome as shown in Figure 1A which is used in place of a logistic function and assume that this provides a component of the improvement in accuracy. The plot in Fig 1B shows that calibration is good as a result. Does this mapping reveal aspects of the score that might be investigated further?

The Freedom-from-Event predictor shows a step up in complexity of modelling. The authors have provided a useful mechanism with which to predict hospital events that could if necessary be calculated 'by hand'. The next model is likely to be even more complex and will need implementation on for example a PDA or mobile phone. This is liberating for modellers since there is no longer any constraint on complexity so that for example interactions and the hierarchical nature of the data can be taken into account, or more advanced classification tools than logistic regression can be employed.

Reference List

1 Brieger D, Fox KAA, Fitzgerald G, et al. Predicting freedom from clinical events in non-ST-elevation acute coronary syndromes. The Global Registry of Acute Coronary Events. Heart Online First 2009.

2 Fox KA, Goodman SG, Klein W, et al. Management of acute coronary syndromes. Variations in practice and outcome; findings from the Global Registry of Acute Coronary Events (). Eur Heart J 2002 Aug;23(15):1177-89.