Article Text
PDF
Abstract
Objective The European Society of Cardiology (ESC) 0/1 hour algorithm has been primarily validated in Europe, America and Australasia with less knowledge of its performance outside of these settings. We aim to evaluate the performance of the ESC 0/1 hour algorithm across different contexts.
Methods We searched PubMed, Embase, Scopus, Web of Science and the Cochrane Central Register of Controlled Trials for relevant studies published between 1 January 2008 and 31 May 2019. The primary outcome was index myocardial infarction and the secondary outcome was major adverse cardiac event or mortality. A bivariate random-effects meta-analysis was used to derive the pooled estimate of each outcome.
Results A total of 11 014 patients from 10 cohorts were analysed for the primary outcome. The algorithm based on high-sensitivity cardiac troponin (hs-cTn)T (Roche), hs-cTnI (Abbott) and hs-cTnI (Siemens) had pooled sensitivity of 98.4% (95% CI=95.1% to 99.5%), 98.1% (95% CI=94.6% to 99.3%) and 98.7% (95% CI=97.3% to 99.3%), respectively. The algorithm based on hs-cTnT (Roche) and hs-cTnI (Siemens) had pooled specificity of 91.2% (95% CI=86.0% to 94.6%) and 95.9% (95% CI=94.1% to 97.2%), respectively. Among patients in the rule-out category, the pooled mortality rate at 30 days and at 1 year was 0.1% (95% CI=0.0% to 0.4%) and 0.8% (95% CI=0.5% to 1.2%), respectively. Among patients in the observation zone, the pooled mortality rate was 0.7% (95% CI=0.3% to 1.2%) at 30 days but increased to 8.1% (95% CI=6.1% to 10.4%) at 1 year, comparable to the mortality rate in the rule-in group.
Conclusion The ESC 0/1 hour algorithm has high diagnostic accuracy but may not be sufficiently safe if the 1% miss-rate for myocardial infarction is desired.
PROSPERO registration number CRD42019142280.
- acute myocardial infarction
- acute coronary syndromes
Statistics from Altmetric.com
Introduction
Chest pain and associated symptoms constitute a significant proportion of the emergency department (ED) visits.1 2 However, only 10%–20% of the patients presenting with these complaints are eventually diagnosed with acute myocardial infarction (AMI).3 Timely diagnosis of AMI is essential because early revascularisation with an evidence-based care bundle can decrease mortality.4
The recent introduction of high-sensitivity cardiac troponin (hs-cTn) assays has allowed the development of several rapid triage protocols, including the European Society of Cardiology (ESC) 0/1 hour algorithm.5 This approach combines a very low hs-cTn at initial ED presentation and dynamic change of hs-cTn values between 0 and 1 hour to triage patients into rule-out, observation and rule-in categories, allowing clinical decision making in just over 2–3 hours.
Even though the ESC 0/1 hour algorithm has been validated in a large number of prospective studies6–12, use of this algorithm has been limited to less than 15% of the hospital institutions globally.13 Because the algorithm has primarily been evaluated in Europe, America and Australasia,6–9 there are some concerns about its safety outside of these settings. Furthermore, the reported diagnostic accuracies seem to be discrepant across different population cohorts.14–16
Previous meta-analyses have focused on the utility of a single hs-cTn test at ED presentation to rule out AMI.17–22 The diagnostic accuracy of the 0/1 hour delta change of hs-cTn, which is central to the ESC algorithm, has not been evaluated. Importantly, none of these studies reported on the prognostic significance of the hs-cTn assays based on the respective triage categories. The outcome of patients that fall into the rule-out and observation group is of great importance for clinical decision making.
To address these knowledge gaps, we conducted a systematic review and meta-analysis to investigate the accuracy estimates and prognostic values of the ESC 0/1 hour algorithm.
Methods
This study was conducted in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies23 and Meta-analyses Of Observational Studies in Epidemiology guidelines.24 The review protocol was registered on the PROSPERO database. Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Search strategy
We searched PubMed and Embase for studies published between 1 January 2008 (when the first hs-cTnI assay was released) and 31 May 2019 using a protocol (online supplementary table 1) that incorporated keywords myocardial infarction, troponin, 0/1 hour algorithm and emergency department. References for original and review articles were manually searched. An updated search on Scopus, Web of Science and the Cochrane Central Register of Controlled Trials was performed to ensure the completeness of data search. There were no restrictions on language. Articles that were published in languages other than English were translated and interpreted using Google translation.
Supplemental material
Study selection
Three reviewers (C-HC, C-HC, GHL) independently screened the titles and abstracts to identify potentially relevant studies. Full texts of relevant publications were screened independently by three reviewers (C-HC, C-HC, GHL). A fourth reviewer (C-CL) confirmed the inclusion or exclusion of the studies.
Eligible studies were prospective cohort studies that evaluated the diagnostic accuracy of the ESC 0/1 hour algorithm in patients presenting to the ED with suspected non-ST elevation myocardial infarction (NSTEMI) or acute coronary syndrome (ACS). Studies that evaluated the high-sensitivity cardiac troponin T (hs-cTnT) (Roche) and high-sensitivity cardiac troponin I (hs-cTnI) (Abbott) assays were included.5 Studies that assessed the hs-cTnI (Diemension Vista) assay were excluded. Studies on other assays that were not yet recommended by the 2015 ESC guidelines were included if there were three or more cohorts available.
Studies should use myocardial infarction as the primary endpoint based on the contemporary universal definition.25 Studies should exclude patients with initial diagnosis of STEMI, or include sub-group data on only patients with NSTEMI. Studies that used major adverse cardiovascular events (MACE) or mortality as the primary endpoint were also included for analysis of the safety profile. Conference papers were only included if there were no previous original articles published. For cohorts with multiple publications that have identical study endpoints and overlapping recruitment periods, only the study with the largest sample size or most complete datasets was included (online supplementary methods).
Data extraction
Three reviewers (C-HC, C-HC, GHL) independently abstracted data on study demographics, and both primary and secondary outcomes. The primary endpoint was index admission AMI based on the universal definition of AMI, and the secondary endpoints were death and occurrence of MACE.
Quality assessment
The methodological quality of included studies was assessed by two reviewers (C-HC and C-HC) using the Quality Assessment of Diagnostic Accuracy Studies-2 tool (online supplementary tables 2 and 3).26 Any discrepancies were resolved through discussion with a third reviewer (GHL).
Data synthesis and analysis
Since the ESC 0/1 hour algorithms categorised patients into rule-out, observation and rule-in groups, we calculated pooled sensitivity, negative predictive value (NPV), and negative likelihood ratios (LR-) for the rule-out group, and pooled specificity, positive predictive value (PPV), and positive likelihood ratios (LR+) for the rule-in group separately. We computed triage efficacy as the proportion of patients that were placed in the respective triage groups. Accuracy estimates were pooled using bivariate random-effects regression models. The between-study variation was quantified by I2 statistics. The presence and effect of publication bias were examined using the Deek’s test.27 We performed a meta-analysis of proportions to compare the rate of adverse events and proportion of patients in different triage groups. We conducted sensitivity analyses based on risks of bias. Continuous variables were described as mean or median with 95% CI or IQR, categorical variables as numbers and percentages. All analyses were calculated by the ‘midas,’ ‘metabias,’ 'Fagani’ and ‘metaprop’ package in STATA V.14.0 or R software V.3.5.3.
Results
Initial electronic database searches yielded 385 hits. After title and abstract screening, 54 studies were eligible for full-text review. After full-text review, a total of 15 studies were included for analysis(figure 1)6 7 9 14–16 28–36.
Flowchart describing the process of study selection. AMI, acute myocardial infarction; hs-cTn, high-sensitivity cardiac troponin.
Characteristics of included studies
Overall, 64% (range, 40% to 70%) of 11 014 participants were male, with a mean age of 61 years (range, 55–72 years). The prevalence of AMI ranged from 2% to 32%, with an overall pooled prevalence of 16%. The time from ED presentation to first sample collection ranged from 14.9 min to 28 min, and the time from first to second sample collection ranged from 60 min to 65 min (table 1). The key characteristics and details of accuracy estimates of all included studies are described in online supplementary tables 4 to 8.
Characteristics of studies included in the systematic review
Quality assessment
All studies prospectively recruited patients presenting to the ED with symptoms suggestive of AMI. Some studies enrolled consecutive or random samples of patients (online supplementary figure 1 and table 3). Most studies used pre-specified cut-off points to classify patients; three studies derived the 0/1 hour algorithm.6 7 29 In all studies, AMI was defined according to the Global Task Force and adjudicated independently by two or three physicians.25 In three studies, researchers who adjudicated the outcomes were blinded to the results of the index test.9 14 33 Six studies that did not employ blinding during the adjudication process also used hs-cTn clinically for adjudication of MI.6 7 28–31 Several studies did not include all eligible patients for analysis because of missing data, most commonly due to missing samples at 1 hour.
Rule-out performance
Based on hs-cTnT (Roche), the algorithm ruled out 55% of the patients as low risk for AMI, with a pooled sensitivity of 98.4% (95% CI=95.1% to 99.5%; I2=29%) and NPV of 99.6% (95% CI=99.0% to 99.9%) (figure 2 and online supplementary table 9). Similarly for hs-cTnI (Abbott) and hs-cTnI (Siemens), the algorithm ruled out more than 50% of the patients as low risk with pooled sensitivities of 98.1% (95% CI=94.6% to 99.3%; I2=25%) and 98.7% (95% CI=97.3% to 99.3%; I2=17%), and NPVs of 99.3% (95% CI=95.9%–99.9%) and 99.6% (95% CI=99.2% to 99.8%), respectively.
Forest plot for acute myocardial infarction and summary estimates for sensitivity and NPV. Note: Reichlin et al 6 (derivation cohort for hs-cTnT (Roche)) set an optimal threshold for a 100% sensitivity and NPV. Rubini et al 7 (derivation cohort for hs-cTnI (Abbott)) set an optimal threshold for a 99% sensitivity and 95% NPV. Boeddinghaus et al 29 (derivation cohort for hs-cTnI (Siemens)) set an optimal threshold for a 99% sensitivity and NPV. APACE, advantageous predictors of acute coronary syndromes evaluation; BACC, biomarkers in acute cardiac care; HIGHSTEACS, high-sensitivity troponin in the evaluation of patients with acute coronary syndrome; HIGH-US, high-sensitivity cardiac troponin I assays in the United States; NPV, negative predictive value; REACTION-US, rapid evaluation of acute myocardial infarction in the United States; TRAPID, the high sensitivity cardiac troponin T assay for rapid rule-out of acute myocardial infarction.
Forest plot for acute myocardial infarction and summary estimates for specificity and PPV. Note: all the three derivation cohorts (Reichlin et al 6, Rubini et al 7 and Boeddinghaus et al 29 defined the optimal thresholds based on a classification and regression tree analysis. Boeddinghaus et al (derivation cohort for hs-cTnI (Siemens)) set an optimal threshold for a 70% PPV. APACE, advantageous predictors of acute coronary syndromes evaluation; BACC, biomarkers in acute cardiac care; HIGHSTEACS, high-sensitivity troponin in the evaluation of patients with acute coronary syndrome; HIGH-US, high-sensitivity cardiac troponin I assays in the United States; PPV, positive predictive value REACTION-US, rapid evaluation of acute myocardial infarction in the United States; TRAPID, the high sensitivity cardiac troponin T assay for rapid rule-out of acute myocardial infarction.
Prognostic outcome of patients in respective triage groups. Data were pooled from three cohorts: APACE, advantageous predictors of acute coronary syndromes evaluation; BACC, biomarkers in acute cardiac care; TRAPID, the high sensitivity cardiac troponin T assay for rapid rule-out of acute myocardial infarction.
Rule-in performance
Based on hs-cTnT (Roche), the algorithm ruled in 18% of the patients as high risk for AMI, with a pooled specificity of 91.2% (95% CI=86.0% to 94.6%; I2=11%) and PPV of 51.3% (95% CI=30.8% to 71.3%) (figure 3 and online supplementary table 10). For hs-cTnI (Siemens), the algorithm ruled in 14% of the patients with a higher specificity at 95.9% (95% CI=94.1% to 97.2%; I2=3%). Hs-cTnI (Abbott) could not be pooled owing to insufficient studies.
Prognosis of patients in different triage groups
For patients in the rule-out group, the pooled 30-day mortality rate was 0.1% (95% CI=0.0% to 0.4%; I2=0%) and increased slightly to 0.8% (95% CI=0.5% to 1.2%; I2=0%) at 1 year (figure 4 and online supplementary table 11). For patients in the observation group, the pooled 30-day mortality rate was 0.7% (95% CI=0.3% to 1.2%; I2=0%), increasing substantially to 8.1% (95% CI=6.1% to 10.4%; I2=0%) at 1 year. For the rule-in group, the pooled 30-day mortality rate was 1.8% (95% CI=0.4% to 4.2%; I2=0%), increasing to 10.0% (95% CI=7.8% to 12.4%; I2=0%) at 1 year.
Sensitivity analysis
In a sensitivity analysis pooling all studies eligible for estimation of the 0/1 hour algorithm, irrespective of assay or quality assessment, the sensitivity of the algorithm in ruling out AMI was 98.3% (95% CI=97.0% to 99.1%) (online supplementary figure 2A), and the specificity of the algorithm in ruling in AMI was 93.4% (95% CI=90.6% to 95.4%) (online supplementary figure 2B). Three cohorts were the derivation cohorts for the 0/1 hour algorithm (one for each assay) and were subjected to high risk of bias. After these cohorts were removed, the remaining validation cohorts had a sensitivity of 98.1% (95% CI=96.3% to 99.0%) (online supplementary figure 2C) and specificity of 93.2% (95% CI=89.2% to 95.7%) (online supplementary figure 2D), comparable to that in the overall pooled analysis.
Discussion
In this meta-analysis, the ESC 0/1 hour algorithm ruled out more than 50% of the patients presenting to the ED with chest pain as low risk for AMI. The algorithm has a good sensitivity and low rates of adverse cardiac events in the rule-out group, and performed similarly across different hs-cTn assays. The algorithm ruled in approximately 15% of the patients with high specificity. A substantial number of patients with unfavourable prognosis were placed in the observation group.
All of the studies in our analysis were prospective, and used the universal definition of myocardial infarction for adjudication of the final diagnosis. Most studies had a low bias towards patient selection. No publication bias was detected. In our sensitivity analysis, we removed derivation studies that would most critically influence the performance of the algorithm. Nevertheless, because many studies used hs-cTn assays as part of the adjudication, or did not use blinding during adjudication, there may be overestimation of the reported accuracy.
In a previous meta-analysis on using hs-cTnT below the limit of detection (LoD) at presentation to rule out AMI, Pickering and associates showed that 30.6% of patients could be ruled out with a pooled sensitivity of 98.7% (95% CI=96.6% to 99.5%).20 In our analysis, the addition of a 0–1 hour change in hs-cTnT levels ruled out more than 50% of the patients with a pooled sensitivity of 98.4% (95% CI=95.1% to 99.5%). The pooled sensitivity of the algorithm was high and comparable across the hs-cTnT (Roche), hs-cTnI (Abbott) and hs-cTnI (Siemens) assays. Nevertheless, the algorithm may not be universally safe if the 1% miss-rate is desired.37 The sensitivity estimate across all assays were less than the consensus goal of 99%. Only five cohorts (APACE, BACC, Japan–Taiwan, Parkland, HIGHSTEACS) reported a sensitivity greater than 99%. Importantly, two cohorts (Barcelona and Fuwai) reported much lower sensitivities (<90%). These observations challenge the universal safety of the algorithm. Given that the pooled sensitivity of the algorithm is below the desired goal of 99%, we suggest that the algorithm be used with caution.
Even though our analysis showed that patients who were ruled out had a favourable short-term and long-term outcome, only three cohorts—mainly cohorts that reported high diagnostic accuracies were represented. Thus, additional investigations are required to ascertain the prognostic outcome of patients ruled out by the algorithm outside of these settings.
Previous systematic reviews that analysed a single 0-hour sample of hs-cTn above the 99th percentile of a normal reference population to rule in AMI reported the specificity to be 77%–82%.17–19 In a recent meta-analysis, using a 50 ng/L cut-off at presentation to rule in AMI resulted in a pooled specificity of 95%.22 In parallel, the 0/1 hour algorithm could rule in patients at a comparable or a higher specificity of 93.6%. In our analysis, the algorithm based on hs-cTnI (Siemens) had higher specificity compared with that on hs-cTnT (Roche). This could be because the hs-cTnT (Roche) assay has been more extensively investigated in geographically diverse cohorts. Cohorts that reported lower specificities (Japan–Taiwan, Parkland and REACTION-US) using the hs-cTnT (Roche) assay were not investigated using the hs-cTnI (Siemens) assay.
The reported specificities were generally high across the cohorts. Only the Japan–Taiwan cohort14 reported a substantially lower specificity (77.5%), presumably because patients with chronic renal disease undergoing dialysis were included in this study. However, the PPV for the algorithm was modest at 51.3%-73.4%. In this case, it may be helpful to discuss work-up strategies with physicians who are involved in patient management,38 because the elevation of troponin often indicates myocardial injury and may require rigorous work-up.39
Our study found that the 0/1 hour algorithm places a substantial proportion of patients in the observation zone, for whom management strategy remains unclear.40 The rates of MACE and mortality of patients in the observation zone were high at 1 year and comparable to those in the rule-in group. A recent study of 15 international cohorts found that patients who did not have AMI but presented with hs-cTn of more than 10 to 14 ng/L—similar to patients in the observation zone, had a three times higher risk of death or AMI compared with the general population.41 These results suggest that patients that were not ruled by the algorithm carry significant mortality risks. Owing to a lack of individual patient-level data, we were unable to describe the composition of this group of patients. However, a multicentre study showed that patients in the observation zone are typically elderly men with pre-existing coronary artery disease and high long-term mortality.42 Clearly, an evidence-based management strategy for this group of patients is urgently needed.
Clinical implications
Recently, the 0/1 hour algorithm has been demonstrated to produce favourable outcomes in real-world implementation. The introduction of the algorithm shortened time to ED discharge and is associated with low rates of adverse cardiac events and mortality.10–12 Our study adds values by demonstrating that the performances of the algorithm based on different hs-cTn assays were comparable. Importantly, this finding may allow institutions to implement hs-cTn testing in patients with suspected AMI, without the financial and logistical challenges of introducing a new assay or analyser exclusively for measuring hs-cTn.
Furthermore, we showed that there are caveats to the algorithm. First, the sensitivity is not universally high across cohorts and may not be sufficiently safe if the 1% miss-rate for myocardial infarction is desired. Second, a significant number of patients were placed in the observation zone, for whom the long-term prognosis is poor. In consideration of these caveats, it is important to reiterate that this troponin-only algorithm should always be used in conjunction with a detailed clinical assessment and the ECG.
Finally, sampling at the algorithm-defined timings, particularly at 1 hour, may be challenging in some EDs. In these instances, it may still be possible to use the 0/1 hour algorithm, as the algorithm has been shown to perform well using samples collected at 2 hours.38 However, because most studies in this analysis did not report sampling times, we were unable to provide evidence on the use of the algorithm outside of these pre-defined timings.
Limitations
There were several limitations to this study. First, we were unable to obtain data of individual patients and could not characterise cases of myocardial infarction missed by the algorithm. Furthermore, the timing of sample collection was not reported in many cohorts. A delayed time from symptom onset to sampling may reduce the potential for missed myocardial infarction and overestimate the algorithm’s sensitivity. Second, the substantial risk of bias in the included studies and the inclusion of derivation cohorts in the meta-analysis may influence the overall meta-estimate. Because most studies used inclusion and exclusion criteria that precluded certain high risk populations, the accuracy estimates reported in this study may differ from those in the real-world settings. Furthermore, some studies were conducted using samples stored for long periods of time, which could be biased by problems of protein degradation and differences in diagnostic procedures.43 44 Third, due to the limited number of studies reporting on patients with chronic kidney disease (CKD), pre-existing coronary artery disease (CAD), or old age, we could not analyse the accuracy of ESC 0/1 hour algorithm on these subgroups. Fourth, because the prevalence of myocardial infarction in our study is substantially higher than that reported by previous epidemiological investigations,45 our study may underestimate the NPV and overestimate the PPV. Fifth, some heterogeneity in the accuracy estimates was noted. A lack of relevant contextual information for different studies prevented the use of meta-regression for adjustment. Lastly, in all studies, patients were not managed according to the algorithm, thus troponin levels and differences in clinical management could have influenced the outcomes.
Conclusion
The ESC 0/1 hour algorithm has high diagnostic accuracy and triage efficacy in ruling out AMI among patients presenting to the ED with symptoms suggestive of myocardial infarction. Different brands of hs-cTn assays show comparable diagnostic accuracy and triage efficacy. A substantial proportion of patients is placed in the observation zone, for whom the prognosis is unfavourable. The algorithm may not be sufficiently safe if the 1% miss-rate for myocardial infarction is desired. Additional investigations on the implementation of the algorithm will provide more insights regarding its safety and clinical applicability.
Key messages
What is already known on this subject?
The European Society of Cardiology (ESC) 0/1 hour algorithm has been demonstrated to triage patients with suspected myocardial infarction with high diagnostic accuracy. However, studies evaluating the performance of the algorithm have reported discrepant results.
What might this study add?
We performed a systematic review and meta-analysis of studies that investigated on the diagnostic performance of the ESC 0/1 hour algorithm. The algorithm had high sensitivity and specificity across different high-sensitivity cardiac troponin assays. A significant proportion of patients were placed in the observation zone, for whom the long-term prognosis was poor.
How might this impact on clinical practice?
Our results support the use of the 0/1 hour algorithm to triage patients with suspected myocardial infarction. However, the algorithm may not be sufficiently safe if the 1% miss-rate for myocardial infarction is desired. Patients in the observation zone have a poor prognosis and management strategies for these patients are urgently needed.
Acknowledgments
We thank the staff of the Core Labs, Department of Medical Research, National Taiwan University Hospital for their technical support.
References
Footnotes
C-HC, C-HC and GHL contributed equally.
Contributors Study concept and design: C-CL and C-HC. Acquisition of data: C-HC, C-HC and GHL. Analysis of data: W-TG and Y-KW. Statistical advice: S-SH and YHY. Drafting of the manuscript: C-CL, C-HC, C-HC and GHL. Critical revision of the manuscript for important intellectual content: W-TG, Y-KW, S-SH, YHY and EG. Study supervision: C-CL and EG.
Funding This work was supported by a research grant from Taiwan National Ministry of Science and Technology Grants MOST 104-2314-B-002 -039 -MY3 and MOST 105-2811-B-002-031.
Competing interests EG reports personal fees from Roche Diagnostics, personal fees from Hoffmann La Roche, personal fees from Brahms Thermo Fischer, personal fees from Daiichi Sankyo, personal fees from Astra Zeneca and personal fees from Bayer Vital outside the submitted work. C-CL reports receiving speaking honorarium from Roche Diagnostics. All other authors report no conflict of interest.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. The authors confirm that the data supporting the findings of this study are available within the article and its supplementary materials.