Article Text

Download PDFPDF

Original research
Comparison of accelerated diagnostic pathways for acute chest pain risk stratification
  1. Jason Stopyra1,
  2. Anna Catherine Snavely2,
  3. Brian Hiestand1,
  4. Brian J Wells3,
  5. Kristin Macfarlane Lenoir2,
  6. David Herrington4,
  7. Nella Hendley1,
  8. Nicklaus P Ashburn1,
  9. Chadwick D Miller1,
  10. Simon A Mahler1
  1. 1 Emergency Medicine, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
  2. 2 Public Health Sciences, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
  3. 3 Translational Science Institute, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
  4. 4 Division of Cardiovascular Medicine, Department of Internal Medicine, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
  1. Correspondence to Dr Jason Stopyra, Emergency Medicine, Wake Forest University School of Medicine, Winston-Salem, NC 27157, USA; jstopyra{at}wakehealth.edu

Abstract

Background The History Electrocardiogram Age Risk factor Troponin (HEART) Pathway and Emergency Department Assessment of Chest pain Score (EDACS) are validated accelerated diagnostic pathways designed to risk stratify patients presenting to the emergency department with chest pain. Data from large multisite prospective studies comparing these accelerated diagnostic pathways are limited.

Methods The HEART Pathway Implementation is a prospective three-site cohort study, which accrued adults with symptoms concerning for acute coronary syndrome. Physicians completed electronic health record HEART Pathway and EDACS risk assessments on participants. Major adverse cardiac events (death, myocardial infarction and coronary revascularisation) at 30 days were determined using electronic health record, insurance claims and death index data. Test characteristics for detection of major adverse cardiac events were calculated for both accelerated diagnostic pathways and McNemar’s tests were used for comparisons.

Results 5799 patients presenting to the emergency department were accrued, of which HEART Pathway and EDACS assessments were completed on 4399. Major adverse cardiac events at 30 days occurred in 449/4399 (10.2%). The HEART Pathway identified 38.4% (95% CI 37.0% to 39.9%) of patients as low-risk compared with 58.1% (95% CI 56.6% to 59.6%) identified as low-risk by EDACS (p<0.001). Major adverse cardiac events occurred in 0.4% (95% CI 0.2% to 0.9%) of patients classified as low-risk by the HEART Pathway compared with 1.0% (95% CI 0.7% to 1.5%) of patients identified as low-risk by EDACS (p<0.001). Thus, the HEART Pathway had a negative predictive value of 99.6% (95% CI 99.1% to 99.8%) for major adverse cardiac events compared with a negative predictive value of 99.0% (95% CI 98.5% to 99.3%) for EDACS.

Conclusions EDACS identifies a larger proportion of patients as low-risk than the HEART Pathway, but has a higher missed major adverse cardiac events rate at 30 days. Physicians will need to consider their risk tolerance when deciding whether to adopt the HEART Pathway or EDACS accelerated diagnostic pathway.

Trial registration number NCT02056964.

  • acute coronary syndromes
  • healthcare delivery

Statistics from Altmetric.com

Introduction

Chest pain is the second most common complaint of patients presenting to an emergency department in the USA, accounting for 5.3% of total presentations and US$10–US$13 billion annual cost.1–3 The History, Electrocardiogram, Age, Risk factors, Troponin (HEART) Pathway and the Emergency Department Assessment of Chest pain Score (EDACS)-Accelerated Diagnostic Protocol (ADP) are validated algorithms used to risk stratify patients presenting to the emergency department with chest pain. Both pathways are used to guide physicians in the identification of low-risk patients with symptoms concerning for possible acute coronary syndrome, who can be safely discharged from the emergency department without stress testing or angiography.

EDACS-ADP was derived in patients from the Asia Pacific Region, demonstrating a sensitivity of 99%–100% for 30-day major adverse cardiac events in urban emergency departments in Australia and New Zealand.4 However, a study evaluating EDACS in a US cohort of patients presenting to the emergency department with suspected acute coronary syndrome demonstrated a sensitivity and negative predictive value of only 88.2% and 98.9%, respectively.5 The HEART Pathway, which is an accelerated diagnostic pathway version of a modified HEART score has demonstrated >99% sensitivity and negative predictive value for major adverse cardiac events within 30 days.6 A recent large retrospective study comparing the accelerated diagnostic pathways suggested that a modified EDACS-ADP identified more patients for early discharge from the emergency department with a similar negative predictive value to a modified HEART Pathway.7 A comparative study from Australia, which retrospectively determined the HEART Pathway risk, demonstrated similar sensitivities and negative predictive value.8 However, to date prospective comparative data evaluating HEART Pathway and EDACS-ADP are very limited. Furthermore, there is some debate among clinicians and experts regarding the acceptable negative predictive value and sensitivity of a chest pain accelerated diagnostic pathway. An international survey of emergency physicians, Than et al, demonstrated that 57% of clinicians desired a negative predictive value ≥99.5% for 30-day major adverse cardiac events.5 Therefore, the objective of this study is to compare the test performance of the HEART Pathway and EDACS-ADP in a large cohort of patients in the USA presenting to the emergency department with chest pain and to determine if either accelerated diagnostic pathway can achieve a negative predictive value of ≥99.5% for 30-day major adverse cardiac events.

Methods

Study design

A planned secondary analysis of participants enrolled in the prospective, three-site HEART Pathway Implementation Study was conducted. Participants for this analysis were accrued from November 2014 through January 2016 under a waiver of informed consent. This study was approved by the Institutional Review Board of the sponsoring organisation. Methods of the HEART Pathway Implementation Study have been previously described.9 There was no patient or public involvement in the design or analysis of this study. However, patient representatives have been involved in dissemination methods.

Study setting

Participants were accrued from the emergency departments of three sites within a health system (name withheld for review) which serves urban, suburban and rural populations. The emergency departments are staffed by board-certified or board-eligible emergency physicians 24 hours per day, 7 days a week, who directly provide care and oversee care provided by residents (at the single academic site), physician assistants and nurse practitioners. Cumulative emergency department patient volume during the enrolment period consisted of approximately 163 000 patient encounters per year.

Participants

All patients aged at least 21 years presenting with a chief complaint of chest pain and at least one troponin ordered, without evidence of ST-segment elevation myocardial infarction on ECG were accrued. In addition, patients with other complaints that were concerning for acute coronary syndrome were included if the physician used a study-specific electronic health record flowsheet for possible acute coronary syndrome. Patients with only one troponin ordered were included because this demonstrated a physician’s basal concern for acute coronary syndrome.

Data collection

Data elements for the HEART Pathway and EDACS-ADP assessments were entered prospectively by physicians into an electronic health records-based clinical decision support tool. The HEART Pathway and EDACS-ADP are represented in figures 1 and 2, respectively. For all patients aged 21 years or older with chest pain and a troponin order, an interruptive pop-up alert appeared in the electronic health record as a best practice advisory. The pop-up prompted physicians to answer a series of questions, within an electronic health record flowsheet for possible acute coronary syndrome, which were necessary for the HEART Pathway and EDACS-ADP assessments. In addition, the clinical decision support tool could also be accessed by manually selecting the electronic health record flowsheet for possible acute coronary syndrome when physicians identified patients with chest pain equivalents (eg, arm or jaw pain, shortness of breath). The ECG was interpreted by the treating physician in the emergency department. Additional index visit data, patient demographics, cardiovascular risk factors and troponin results were extracted from the health system’s electronic health record data warehouse (Clarity-Epic Systems, Verona, Wisconsin, USA). Serum troponin measurements were performed using the ADVIA Centaur platform TnI-Ultra assay (Siemens, Munich, Germany) or the Access AccuTnI+3 assay (Beckman Coulter, Brea, California, USA). The 99th percentile upper reference limit for each assay was used as the cut-point for a positive troponin measure. Major adverse cardiac event outcomes were determined using validated International Statistical Classification of Diseases (ICD)-9 and ICD-10 codes, from the health system electronic health records, insurers’ claim data (Blue Cross Blue Shield, MedCost and Medicaid) or query of the state death index.

Figure 1

Emergency Department Assessment of Chest pain Score-Accelerated Diagnostic Protocol (EDACS-ADP).

Figure 2

History Electrocardiogram Age Risk factor Troponin (HEART) Pathway. HEAR, History Electrocardiogram Age Risk factor.

Outcomes

The primary outcome was major adverse cardiac events within 30 days of presentation, defined as the composite of all-cause death, acute myocardial infarction or coronary revascularisation. The definition of major adverse cardiac events was based on the standardised reporting guidelines for studies evaluating risk stratification of patients presenting to the emergency department with potential acute coronary syndrome.10 Acute myocardial infarction and coronary revascularisation were determined using diagnosis and procedure codes validated by prior cardiovascular trials.11–13 Coronary revascularisation was defined as coronary artery bypass grafting, stent placement or other percutaneous coronary intervention. Secondary outcomes included the composite of death or myocardial infarction through 30 days and the proportion of patients identified as low-risk (suitable for early discharge).

Data analysis

The sensitivity, specificity, positive predictive value, negative predictive value and positive and negative likelihood ratios (LR) for 30-day major adverse cardiac events were calculated. Corresponding exact binomial 95% CIs were computed for sensitivity, specificity, positive predictive value and negative predictive value. For the positive and negative LR, 95% CIs were calculated using the method by Simel et al.14 Based on an international survey, which found that most physicians are uncomfortable with a ≥0.5% miss-rate for major adverse cardiac events,15 a goal for this analysis was set a priori for each accelerated diagnostic pathway to reach a negative predictive value ≥99.5% for the detection of major adverse cardiac events at 30 days. These procedures were repeated using the secondary composite outcome of 30-day death or myocardial infarction. The percentage of patients identified by the HEART Pathway and EDACS as low risk (safe for early discharge) was determined and reported along with 95% exact binomial CIs. McNemar’s tests were used to compare accelerated diagnostic pathway outcomes such as per cent low-risk and sensitivity. A generalised score statistic proposed by Leisenring et al was used to compare negative predictive values.16 Consistent with prior accelerated diagnostic pathway studies, patients with incomplete follow-up (12.4%, 545/4399) were considered free of 30-day major adverse cardiac events for the primary analysis.6 17 A sensitivity analysis using only those patients with complete follow-up was carried out to support this assumption (online supplementary appendix 1). An additional sensitivity analysis was performed using only patients with two troponin tests (online supplementary appendix 2). The sample size was fixed as this was a secondary analysis of a prospectively collected data set; formal power analysis was therefore not performed. Statistical analysis was performed using R V.3.5.1 (www.R-project.org).

Supplemental material

Supplemental material

Results

From November 2014 to January 2016, 5799 patients presenting to the emergency department with symptoms suggestive of acute coronary syndrome were accrued, of which both HEART Pathway and EDACS physician risk assessments were completed on 4399 (figure 3). Cohort characteristics are summarised in table 1. Major adverse cardiac events at 30 days occurred in 449/4399 (10.2%; 409 index, 40 postdischarge), including 41/4399 (0.9%; 15 index, 26 postdischarge) with death, 377/4399 (8.6%; 368 index, 9 postdischarge) with myocardial infarction, and 42/4399 (1.0%; 31 index, 11 postdischarge) with coronary revascularisation without myocardial infarction. The frequency of EDACS and HEART Pathway determinants are in tables 2 and 3, respectively. Serial troponin measurement was delayed (≥4 hours) in 33.1% (1456/4399) of this cohort.

Figure 3

Flow diagram. MI, Myocardial Infarction; WFBMC, Wake Forest Baptist Medical Center; DMC, Davie Medical Center; LMC, Lexington Medical Center; EHR, Electronic health records; ACS, Acute coronory syndrome; HEART, History Electrocardiogram Age Risk factor Troponin; EDACS, Emergency Department Assessment of Chest pain Score ; MACE, Major Adverse Cardiac Events.*Deaths not exclusive of MI or coronory revascularisation events.

Table 1

Cohort characteristics

Table 2

Frequency of Emergency Department Assessment of Chest pain Score determinants

Table 3

Frequency of History Electrocardiogram Age Risk factor Troponin (HEART) Pathway determinants

The HEART Pathway was 98.4% (95% CI 96.8% to 99.4%) sensitive for major adverse cardiac events with a negative predictive value of 99.6% (95% CI 99.1% to 99.8%) compared with EDACS, which was 94.2% (95% CI 91.6% to 96.2%) sensitive for major adverse cardiac events, with a negative predictive value of 99.0% (95% CI 98.5% to 99.3%). The performance characteristics for EDACS and the HEART Pathway are summarised in table 4 for 30-day major adverse cardiac events. Both sensitivity (p<0.001) and negative predictive value (p=0.003) were significantly improved using the HEART Pathway. For death or myocardial infarction through 30 days, the HEART Pathway was 98.5% (95% CI 96.8% to 99.5%) sensitive compared with 96.6% (95% CI 94.3% to 98.1%) for EDACS (p=0.02). However, the negative predictive value for death or myocardial infarction was 99.6% (95% CI 99.2% to 99.9%) for the HEART Pathway and 99.5% (95% CI 99.1% to 99.7%) for the EDACS-ADP (p=0.21). Additional performance characteristics for EDACS and the HEART Pathway for the composite of death or myocardial infarction through 30 days are reported in table 5.

Table 4

Performance characteristics of the HEART Pathway vs EDACS for 30-day MACE

Table 5

Performance characteristics of the HEART Pathway vs EDACS for 30-day death/MI

The HEART Pathway identified 38.4% (1691/4399, 95% CI 37.0% to 39.9%) of patients as low risk compared with 58.1% (2556/4399, 95% CI 56.6% to 59.6%) identified as low risk by EDACS (p<0.001). Major adverse cardiac events occurred in 0.4% (7/1691, 95% CI 0.2% to 0.9%; 4 deaths, 2 myocardial infarctions, 1 revascularisation without myocardial infarction) of patients classified as low risk by the HEART Pathway compared with 1.0% (26/2556, 95% CI 0.6% to 1.5%; 6 deaths, 8 myocardial infarctions, 12 revascularisations without myocardial infarction) of patients identified as low risk by EDACS (p≤0.001).

Discussion

In this prospective comparison of HEART Pathway and EDACS-ADP assessments, the HEART Pathway was found to be more sensitive than the EDACS-ADP for identifying 30-day major adverse cardiac events in a cohort of patients presenting to the emergency department with chest pain in the USA. While the EDACS-ADP identified more patients as low risk, 1.0% of them had 30-day major adverse cardiac events. The HEART Pathway identified fewer patients as low risk, but only 0.4% of them experienced major adverse cardiac events at 30 days. The acceptable missed major adverse cardiac event rate is an ethical question that continues to be debated, but most physicians are uncomfortable with a negative predictive value <99.5%0.15 Thus, in our cohort the EDACS-ADP had a negative predictive value for major adverse cardiac events that many clinicians consider unacceptable.

In the original EDACS-ADP derivation cohort, 42.2% of patients with acute chest pain were identified as safe for early discharge while maintaining 99% sensitivity for major adverse cardiac events.4 The data presented here yields a lower point estimate of sensitivity of the EDACS-ADP (94.2%), but somewhat higher than our analysis from a previous US cohort (88.2%).5 In this study and our prior study, most of the missed events in patients identified as low risk by the EDACS-ADP were coronary revascularisation events. Prior studies by Than et al of patients with chest pain in the Asia Pacific Region report low coronary revascularisation rates.18 19 However, revascularisation procedures are more common in the USA compared with other high-income countries.20 21 These differences highlight the difficulty in generalising the results of chest pain risk stratification studies conducted in other healthcare systems to US healthcare systems.

In this study, 12 patients who were identified as low risk by EDACS-ADP underwent coronary revascularisation without a myocardial infarction. The importance and appropriateness of revascularisation events without myocardial infarction is a matter of considerable debate. Many believe coronary revascularisation is a treatment made subjectively by a clinician and not an objective, adverse outcome.22 23 However, for this analysis, we selected major adverse cardiac events (death, myocardial infarction and coronary revascularisation) as the a priori primary composite end point because the original EDACS-ADP and HEART Pathway research included these outcomes.4 6 However, if revascularisation events are ignored and only death and myocardial infarction are considered adverse events the performance of the EDACS-ADP improves and meets the negative predictive value threshold of 99.5%. In fact, the negative predictive value of the HEART Pathway and EDACS-ADP for 30-day death and myocardial infarction are similar.

In today’s healthcare climate, safely and effectively using resources are paramount. Accelerated diagnostic pathways such as the HEART Pathway and EDACS have the dual mission of identifying at-risk patients while also minimising costs by reducing the need for in-hospital evaluations of patients with low-risk chest pain.24 This study shows that EDACS identifies more patients as safe for early discharge, which may lead to substantial healthcare system savings. However, clinicians, healthcare systems and society may not accept the benefit of these savings against the higher major adverse cardiac event rate associated with EDACS when choosing an accelerated diagnostic pathway to adopt. This dilemma will likely continue to be debated, but requires sound evidence and appropriate accelerated diagnostic pathway use. It is important for healthcare professionals to understand each accelerated diagnostic pathway’s criteria, test characteristics for adverse outcomes and how to apply it appropriately to achieve the level of sensitivity and negative predictive value desired.

Limitations

Our analysis has limitations. The Australasian cohorts in which EDACS was derived and validated had a slightly higher major adverse cardiac event rate than our cohort (13% vs 10.2%). However, conducting this study within a lower-risk cohort may add external validity to our findings since accelerated diagnostic pathways are much less likely to be used by US physicians in patients with obvious acute coronary syndrome or known coronary artery disease. Another potential limitation of this analysis is incomplete follow-up on 545 patients (12.4% of participants), which may have caused misclassification and underestimation of major adverse cardiac events. However, none of these patients appeared in the state death index. While most patients had troponin measurements obtained at 0 and 3 hours after arrival, nearly a third of patients had their second troponin measure at ≥4 hours. This is longer than the serial interval of 0 and 2 hours used in the derivation and validation of the EDACS-ADP. However, an increased serial troponin interval would be expected to enhance rather than harm sensitivity and negative predictive value.25 26 Furthermore, this study did not use high sensitivity troponin assays, therefore the performance of the HEART Pathway and EDACS-ADP in combination with the high-sensitivity troponin assays in this cohort is unclear. In addition, the original EDACS-ADP derivation/validation manuscript includes a footnote that patients with crescendo angina should be excluded from the low-risk group. However, the authors do not provide an objective definition of crescendo angina. In the absence of a clear definition, the HEART Pathway Implementation study and this analysis did not include a crescendo angina variable. Finally, our results may have been susceptible to a workup bias since only the HEART Pathway was used to guide clinical care rather than EDACS-ADP. This may have resulted in low-risk EDACS patients receiving more inpatient cardiovascular investigations and greater opportunity to be diagnosed with major adverse cardiac events.

Conclusion

In this prospective multisite comparison of the HEART Pathway and EDACS-ADP assessments, the HEART Pathway was more sensitive than the EDACS-ADP for identifying 30-day major adverse cardiac events in a US cohort of patients presenting to the emergency department with chest pain. EDACS-ADP identified a larger proportion of patients as low risk, but 1.0% of these low-risk patients had 30-day major adverse cardiac events. The HEART Pathway identified fewer patients as low risk, but only 0.4% of them experienced major adverse cardiac events at 30 days. While the acceptable rate of missed major adverse cardiac events has been debated, an international survey found that most physicians are uncomfortable with a negative predictive value of <99.5%. However, if only death and myocardial infarction are considered adverse events (excluding coronary revascularisation) then our results support use of the EDACS-ADP. Emergency department physicians, groups and health systems will need to consider their risk tolerance versus their desire to discharge more patients with chest pain when deciding whether to adopt the HEART Pathway or EDACS-ADP.

Key questions

What is already known on this subject?

  • The History Electrocardiogram Age Risk factor Troponin (HEART) Pathway and Emergency Department Assessment of Chest pain Score (EDACS) accelerated diagnostic pathways are validated algorithms used to risk stratify patients presenting to the emergency department with chest pain.

  • Both pathways are designed to identify low-risk patients with acute chest pain, who can be safely discharged from the emergency department without stress testing or angiography.

What might this study add?

  • In a US cohort of 4399 patients with a 10.2% major adverse cardiac event rate at 30 days, EDACS identified a larger proportion of patients as low risk than the HEART Pathway, but had a higher missed major adverse cardiac event rate at 30 days.

How might this impact on clinical practice?

  • Physicians, groups and health systems should consider their risk tolerance when deciding whether to adopt the HEART Pathway or EDACS accelerated diagnostic pathway.

  • Prior data on physicians’ risk tolerance suggest that many physicians will find EDACS’ 1% missed major adverse cardiac event rate to be unacceptably high.

Acknowledgments

Gregory L. Burke MD, MSc; Pamela W. Duncan PhD; L. Douglas Case PhD; David M. Herrington MD, MHS; Jose-Franck Diaz-Garelli PhD; Wendell M. Futrell BS; William B. Applegate MD, MPH; Robert F. Riley MD, MS; Erin N. Harper MS; Stephanie Elliott; Russell M. Howerton MD and Bob Mckee provided valuable assistance and support with implementation of the HEART Pathway.

References

Footnotes

  • Twitter @Nick_Ashburn1

  • Contributors All authors contributed to the planning, conduct and reporting of the work described in this article. JS and SAM are responsible for the overall content as guarantors.

  • Funding The HEART Pathway Implementation Study was funded by the Donaghue Foundation.

  • Competing interests SM also receives research funding/support from Abbott Point of Care, Roche Diagnostics, Siemens, PCORI and NHLBI (1 R01 HL118263-01, L30 HL120008). SM is the Chief Medical Officer for Impathiq Inc. JS receives research funding/support from Abbott Point of Care, Roche Diagnostics and and NHLBI (1 R01 HL118263-01). CM receives research funding/support from Siemens, Abbott Point of Care and 1 R01 HL118263. ACS receives research funding from NHLBI (1 R01 HL118263-01).

  • Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.

  • Patient consent for publication Not required.

  • Ethics approval This study was approved by the Institutional Review Board of the sponsoring organisation.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement Data are available on reasonable request. Data that support the findings of this study are available on request from the corresponding author (http://orcid.org/0000-0001-7457-3969). The data are not publicly available due to information that could compromise the identity of research participants.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles