Background Risk-benefit for cardiac resynchronisation therapy (CRT) defibrillator (CRT-D) over CRT pacemaker remains a matter of debate. We aimed to identify patients with a poor outcome within 1 year of CRT-D implantation, and to develop a CRT-D Futility score.
Methods Based on an administrative hospital-discharge database, all consecutive patients treated with prophylactic CRT-D implantation in France (2010–2019) were included. A prediction model was derived and validated for 1-year all-cause death after CRT-D implantation (considered as futility) by using split-sample validation.
Results Among 23 029 patients (mean age 68±10 years; 4873 (21.2%) women), 7016 deaths were recorded (yearly incidence rate 7.2%), of which 1604 (22.8%) occurred within 1 year of CRT-D implantation. In the derivation cohort (n=11 514), the final logistic regression model included—as main predictors of futility—older age, diabetes, mitral regurgitation, aortic stenosis, history of hospitalisation with heart failure, history of pulmonary oedema, atrial fibrillation, renal disease, liver disease, undernutrition and anaemia. Area under the curve for the CRT-D Futility score was 0.716 (95% CI: 0.698 to 0.734) in the derivation cohort and 0.692 (0.673 to 0.710) in the validation cohort. The Hosmer-Lemeshow test had a p-value of 0.57 suggesting accurate calibration. The CRT-D Futility score outperformed the Goldenberg and EAARN scores for identifying futility. Based on the CRT-D Futility score, 15.9% of these patients were categorised at high risk (predicted futility of 16.6%).
Conclusions The CRT-D Futility score, established from a large nationwide cohort of patients treated with CRT-D, may be a relevant tool for optimising healthcare decision-making.
- Defibrillators, Implantable
Data availability statement
Data are available on reasonable request.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Cardiac resynchronisation therapy (CRT) and implantable cardioverter defibrillators (ICDs) are recommended, in primary prevention, in patients with heart failure and left ventricular systolic function ≤35%, wide QRS and on optimal drug treatment, to reduce the risk of all-cause and cardiac death and to improve symptoms and quality of life.1 2 Several randomised trials have shown that CRT improves left ventricular systolic function and reduces overall mortality and sudden cardiac death (SCD).1 Many studies have also shown that ICD alone reduces the risk of SCD in systolic heart failure.2 However, optimal criteria to better identify patients who will benefit most from an ICD in addition to CRT are still being debated, and the benefits of ICD may depend on age and underlying cardiac disease.3 Previous studies have proposed different risk scores, but there is a continuing debate on whether to implant a CRT defibrillator (CRT-D) or a CRT pacemaker (CRT-P) in some patients. The choice between CRT-D and CRT-P remains an issue in daily practice, both clinically and economically.4 Few studies have tried to compare outcomes between patients implanted with a CRT-P or CRT-D,5 and there is no strong criterion to help guide the choice between CRT-P and CRT-D in patients with heart failure.
The objective of this study was to identify patients likely to derive little benefit from CRT-D implantation. Based on the French national hospitalisation database, we sought to identify patients with a poor outcome relatively early after CRT-D implantation (ie, all-cause death during the year after implantation) and to develop a prediction model and calculator for identifying these patients.
This longitudinal cohort study was based on the national hospitalisation database covering hospital care for the entire French population. The data for all patients admitted for prophylactic CRT-D implantation in France from January 2010 to December 2019 were collected from the national administrative French Programme de Médicalisation des Systèmes d’Information (PMSI) database. It includes more than 98% of the French population (67 million people) from birth (or immigration) to death (or emigration), even if a person changes occupation or retires. This process allows the determination of each hospital’s budget in the 1546 French healthcare facilities for both public and private hospitals. Each hospitalisation is encoded in a standardised dataset, which includes information about the patient (age and sex), length of hospitalisation pathologies and procedures. Routinely collected medical information includes the principal diagnosis and secondary diagnoses coded according to the International Classification of Diseases, 10th Revision (). All medical procedures are recorded according to the national nomenclature, Classification Commune des Actes Medicaux (online supplemental table 1). The reliability of PMSI data has been validated,6 and this database has previously been used to study patients with cardiovascular conditions, including those treated with CRT.7
The study was conducted retrospectively and, as patients were not involved in its conduct, there was no impact on their care. Ethical approval was not required as all data were anonymised. Procedures for data collection and management were approved by the Commission Nationale de l'Informatique et des Libertés, the independent National Ethical Committee protecting human rights in France, which ensures that all information is kept confidential and anonymous (authorisation no. 1897139).
Patient information (demographics, comorbidities, medical history, procedures and events during hospitalisation or follow-up) was described using data collected from the hospital records. Each variable was identified using ICD-10 codes. For the purpose of our analysis, we excluded all patients with <1 year of follow-up data (except those who died in the first year of follow-up) and patients implanted in secondary prevention. The overall sample of 23 029 patients with CRT-D was randomly partitioned into derivation and validation populations. A prediction model was then derived and validated.
Patients were followed until 31 December 2019 for the occurrence of outcomes. We aimed to evaluate the incidence of all-cause death, from the date of CRT-D implantation until the date of death or last follow-up in the absence of death. According to European Society of Cardiology guidelines,1 1 year of life expectancy is required to implant a defibrillator. Therefore, 1-year death was defined as CRT-D futility and was the study primary outcome. A prediction model was then derived and validated.
Qualitative variables are described as counts and percentages and quantitative variables as means±SDs. Comparisons were made using the χ2 tests for categorical variables and Student’s t-test or the non-parametric Kruskal-Wallis test, as appropriate, for continuous variables. The characteristics of patients who died in the year after the procedure were compared with those who were alive at 1 year. Multivariable logistic regression was used, and the results expressed as ORs and 95% CIs. The CRT-D Futility score was built using multivariable logistic regression. To create the score points, the regression coefficients with p<0.01 were divided by the smallest coefficient and rounded to the nearest integer.8 9 Receiver operating characteristic curves were constructed and Harrell’s C-indexes (ie, area under the curve (AUC)) were calculated as a measure of model performance. A proxy of the Goldenberg score and EAARN (Ejection fraction, Age, Atrial fibrillation, Renal dysfunction, New York Heart Association (NYHA) class IV) score was then calculated. NYHA class was not directly available, but we considered that patients hospitalised with a diagnosis of congestive heart failure had NYHA class III and those with a diagnosis of pulmonary oedema or cardiogenic shock had NYHA class IV.10 11 Left ventricular ejection fraction was not available in the dataset and was thus omitted. Receiver operating characteristic curves with the Goldenberg score and EAARN were compared using the DeLong test, in the derivation and validation cohorts. Decision-curve analysis was used to quantify the clinical usefulness of the prediction models, in the derivation and validation cohorts. Model calibration was considered acceptable at the Hosmer-Lemeshow goodness-of-fit test p>0.1. We then plotted observed versus predicted risks by decile of predicted risk, and the regression line was compared against the line of equality (intercept=0, slope=1), in the derivation and validation cohorts. The incidence rates (%/year) for each outcome during follow-up were then estimated in groups of interest. To quantify the clinical usefulness of the prediction models, we assessed the categorical net reclassification improvement (NRI) of the different scores. All comparisons with p<0.05 were considered statistically significant. Analyses were performed using Enterprise Guide 7.1 (SAS Institute, Cary, North Carolina, USA) and STATA V.16.0 (Stata, College Station, Texas, USA).
A total of 36 015 adults (mean age 72±11 years, including 9400 (26.1%) women; online supplemental table 2) were hospitalised for a CRT-P or CRT-D implantation from 1 January 2010 through 31 December 2019. Of these, 23 029 underwent CRT-D implantation (mean age 68±10 years, including 4873 (21.2%) women) and comprise the study population (online supplemental figure 1). The characteristics of patients with CRT-D in the derivation (n=11 514) and validation (n=11 515) cohorts are presented in online supplemental table 3. Mean±SD follow-up was 4.2±2.5 years (median 3.9, IQR 2.1–6.1) and 7016 deaths were recorded (yearly incidence rate 7.2%), among which 1604 (22.9%) occurred in the year after CRT-D implantation (7.0% of all patients with CRT-D in our study). In the derivation cohort, patients with all-cause death during the first year of follow-up were older and more frequently men (table 1). They also had higher rates for most comorbidities except obesity.
Predictors of futility
The prediction model was developed in the derivation cohort. The final logistic regression model included the following as the main predictors of futility: older age, diabetes, mitral regurgitation, aortic stenosis, history of hospital stay with heart failure, history of pulmonary oedema, atrial fibrillation, renal disease, liver disease, undernutrition and anaemia. Left bundle branch block (compared with CRT-D with no left bundle branch block) was the only predictor of lower risk of early death during the first year of follow-up. Sex, coronary artery disease, vascular disease, dyslipidaemia and sleep apnoea syndrome were significantly associated with 1-year mortality in patients treated with CRT-D (table 2), but were not included in the model as only regression coefficients with p<0.01 were included. The model is presented in table 2, with corresponding points for each variable.
The AUC for the CRT-D Futility score was 0.692 in the validation cohort, outperforming both the Goldenberg score (AUC: 0.637; p<0.0001 for the DeLong test) and the EAARN score (AUC: 0.667; p<0.0001 for the DeLong test) (figure 1, upper panel and online supplemental table 4). Results were similar in the derivation cohort, in which the AUC for the CRT-D Futility score was 0.716, and similarly outperformed the Goldenberg and EAARN scores (online supplemental figure 2, upper panel and online supplemental table 4). Decision-curve analysis demonstrated that the CRT-D Futility score had the best clinical usefulness of the three risk scores tested for identifying death occurring in the year after CRT-D implantation in the validation cohort (figure 1, lower panel) and in the derivation cohort (online supplemental figure 2, lower panel).
The Hosmer-Lemeshow goodness-of-fit test had a p value of 0.57, suggesting that the model was accurate. The observed versus predicted risk deciles of all-cause death during the first year after CRT-D implantation are shown in figure 2, lower right panel (validation cohort) and in online supplemental figure 3, lower right panel (derivation cohort). Calibration of the CRT-D Futility score was satisfactory across several plots, with a predicted rate of early death after CRT-D implantation ranging from 0% to 30%.
As with the Goldenberg and EAARN scores, we further divided the model arbitrarily for the CRT-D Futility score into four groups, with 1.7%, 3.9%, 8.1% and 16.6% futility. The low-risk group consisted of 16.5% of the patients and the high-risk group consisted of 15.9% of the patients (table 3). The performance of the different risk strata is shown in figure 3 and online supplemental table 5 with high specificity for the high-risk (≥12 points) strata. Survival curves for the four proposed groups with each score are shown in figure 4.
The OR for all-cause death at 1 year stratified by CRT-D Futility score indicated a decreasing benefit for patients treated with CRT-D versus CRT-P with increasing CRT-D Futility score (OR: 0.58, 95% CI: 0.51 to 0.66 for those with CRT-D Futility score ≤8 vs 0.79, 95% CI: 0.71 to 0.88 for those with a score ≥12) (online supplemental figure 4).
Concerning the predictive value of the model by sex in the derivation cohort, the AUC (95% CI) for the CRT-D Futility score was 0.721 (0.673 to 0.768) in women and 0.713 (0.694to 0.732) in men (p=0.76 for the comparison between the two groups). Similar results were found in the validation cohort and in the total population in women versus men: 0.688 (0.668 to 0.708) versus 0.704 (0.658 to 0.750) (p=0.54) and 0.701 (0.687 to 0.715) versus 0.712 (0.679 to 0.745) (p=0.53), respectively.
NRI was calculated for patients who experienced all-cause death within the first year after CRT-D implantation (n=1604) and for those who remained alive (n=21 425). We found that the CRT-D Futility score had the best predictive value for identifying futility compared with the other scores. Considering the CRT-D Futility score versus the Goldenberg score, the NRI was 10.5% (p<0.0001). Regarding the CRT-D Futility score versus the EAARN score, the NRI was 4.8% (p<0.0001) (online supplemental table 6).
Based on our large national database, we developed a new clinical score that accurately identified patients with a high risk of death within 1 year of prophylactic CRT-D implantation, indicating potential futility of the ICD device, with good calibration. A CRT-D Futility score ≥12 was associated with 1-year and 5-year death rates as high as 16.6% and 48.8%, whereas 96.8.4% of patients with a score of <8 were alive at 1 year. The CRT-D Futility score outperformed commonly used scores and gives the opportunity to more accurately select candidates for CRT-D implantation.
The process of shared decision-making between clinicians and patients is particularly important when choosing between CRT-P and CRT-D.1 When considering CRT-D implantation, especially in older adults, one should consider quality of life as well as longevity and cost. Shocks, whether appropriate or inappropriate, may lead to post-traumatic stress syndrome, anxiety disorders and depression. ICD functions may also interfere with the natural process of dying peacefully.12 This has to be considered and explained to the patient when choosing between CRT-D and CRT-P. To do so, practical, clinical scores evaluating all-cause mortality after CRT-D implantation may help the clinician to share information with the patient, to help them understand the risks, benefits and possible consequences of CRT-D or CRT-P implantation and help provide the best individualised treatment.
Advanced heart failure is a known predictor of death due to heart failure progression rather than SCD.10 13 The accuracy of the CRT-D Futility score may be explained by the incorporation of strong clinical markers of advanced heart failure (ie, history of hospitalisation with congestive heart failure or pulmonary oedema).14 As opposed to the commonly used NYHA class and left ventricular ejection fraction criteria, they are more objective, being less influenced by patient or physician evaluation.15 Anaemia and diabetes were included into the score and, as markers of acute heart failure,16 17 may have enhanced the predictive value of the score for poor 1-year survival. The response to CRT influences survival in patients with heart failure,1 18 explaining why left bundle branch block has a protective value whereas atrial fibrillation is a negative factor in the score.
Kidney dysfunction is the only comorbidity considered in the other scores of ICD futility.10 11 However, the CeRtiTuDe study19 showed that the higher rate of death in patients with CRT-P was due to non-cardiac death. The CRT-D Futility score incorporates markers of frailty,14 20 including older age and undernutrition, along with non-cardiac diseases (kidney and liver diseases and diabetes).
Women accounted for 4873 patients (21.2%) of our study population, which makes our study one of the largest to evaluate CRT-D in women. However, this proportion of women receiving CRT-D is similar to other studies.21 This might be because women more often have heart failure with preserved or moderately reduced ejection fraction and are thus less likely to be referred for a CRT-D. However, further subgroup analysis confirmed that this score is still relevant to women.
The CRT-D Futility score, although it outperformed the other scores, was imperfect, as shown by the AUC of 0.716. Predicting death is difficult, as shown by similar AUCs with other scores used in medicine.22 23 This suggests that exceeding this threshold will probably be difficult using classical methods if one wants to remain reasonable in terms of variables included. One objection may be the high number of criteria included in the score, which could limit its practical application compared with other scores. However, a dedicated risk calculator could easily be created, as commonly used in other clinical settings. Accepting increased technological complexity, there may be a case for artificial intelligence and machine learning (including imaging, biological and electrocardiographic parameters with a higher granularity than those in the present analysis), which are not yet available in daily practice.24 Although artificial intelligence could be an interesting tool in the future, one may agree that there is still a role for simple tools and clinical judgement. In this respect, no score will provide a ‘go/no-go’ criterion, yet the CRT-D Futility score may provide a more objective tool than other available scores for helping physicians to select CRT-P or CRT-D devices at the time of implantation. Randomised studies are ongoing, for example, RESET-CRT (NCT03494933), to demonstrate that CRT-P is non-inferior to CRT-D (in terms of all-cause mortality) in patients with chronic heart failure. Scores such as the one developed here may also help us to understand, at least partly, why CRT-D is not superior to CRT-P.25
Several limitations to our work must be acknowledged. A main limitation is inherent to the retrospective, observational nature of the study and its potential biases. This limitation may be partly counterbalanced by the large number of patients studied.
Further, the study was based on administrative data, with limitations inherent to such methodology. Data were not systematically externally checked, with potential for information bias. Some secondary diagnosis may have been under-reported due to a lack of incentive for the hospital, while others may have been misclassified. However, ICD-10 is considered reliable for identifying many of the relevant comorbidities for our analysis.26 27 As coding of complications is linked to reimbursement and is regularly checked from an administrative perspective, it is expected to be of good quality.28 Reassuringly, patients in our study showed prevalences of chronic diseases close to those reported in other observational studies.29 The indication of CRT was left to the discretion of the physicians, as well as device functionality and programming changes on follow-up, and the large European CRT Survey II also suggested that patients treated with CRT in France had similar characteristics to those from other European countries.30 Therapies delivered by CRT-D were not available in this analysis. This point does not constitute a major limitation as ICD therapies are not an accurate surrogate endpoint for SCD and that endpoint is not common to both groups. Furthermore, additional quality-of-life-related data cannot be evaluated using an administrative registry. Further clinical studies are expected to answer this relevant question.
Only in-hospital events were included in the current study, and we had no outpatient data. Results apply to inpatients with CRT-D and may not apply to all those with a CRT-D. However, the goal of this study was to provide a simple clinical approach and a global picture at a national level at the time of implantation of a CRT device. Our analysis was restricted to the variables reported in the database, which meant that characteristics such as ejection fraction, glomerular filtration rate, QRS duration, lead position, medical treatment, biomarkers or device programming were not available for analysis. Drug therapies were not included, which may have affected event rates. However, the size of the study population, including every hospitalisation in France, with a negligible risk of follow-up loss may compensate for some of these limitations.
The CRT-D Futility score was developed and validated from a large national database of prophylactic CRT-D recipients and is based on 1-year all-cause death. The risk prediction model proposed in this study provides accurate prognostic information regarding all-cause death at 1 year after CRT-D implantation. This score outperformed other scores commonly used in ICD candidates, which makes it a reliable tool for helping physicians to select CRT-P or CRT-D devices at the time of implantation.
What is already known on this subject?
Risk-benefit for cardiac resynchronisation therapy (CRT) defibrillator (CRT-D) over CRT pacemaker (CRT-P) remains a matter of debate.
What might this study add?
Area under the curve for the CRT-D Futility score was 0.716 (95% CI: 0.698 to 0.734) in the derivation cohort and 0.692 (0.673 to 0.710) in the validation cohort. The CRT-D Futility score outperformed the Goldenberg and EAARN (Ejection fraction, Age, Atrial fibrillation, Renal dysfunction, New York Heart Association class IV) scores for identifying futility. Based on the CRT-D Futility score, 15.9% of these patients were categorised at high risk (predicted futility of 16.6%).
How might this impact on clinical practice?
The CRT-D Futility score, established from a large nationwide cohort of patients treated with CRT-D, outperformed other scores commonly used in implantable cardioverter defibrillator candidates, making it a reliable tool for helping physicians to select CRT-P or CRT-D devices at the time of implantation.
Data availability statement
Data are available on reasonable request.
Patient consent for publication
Contributors BM: Conceptualisation; investigation; methodology; writing—original draft; writing—review and editing. ABi, ABo: Formal analysis; writing—review and editing. JH: Data curation. BP, FF: Writing—review and editing. VK: Data curation. J-CD: Conceptualisation; investigation; methodology; writing—review and editing. LF: Conceptualisation; investigation; methodology; writing—original draft; writing—review and editing, overall content as guarantor.
Funding Sophie Rushton-Smith, PhD (MedLink Healthcare Communications, London), provided editorial assistance in the preparation of the manuscript (limited to editing for style, referencing, and figure and table editing) and was funded by the authors.
Competing interests LF reports being a consultant or speaker for Bayer, BMS/Pfizer, Boehringer Ingelheim, Medtronic and Novartis.
Patient and public involvement Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.