Objective: To develop a comprehensive and easily applicable prognostic model predicting mortality risk in patients with moderate to severe heart failure.
Design: Prospective follow up study.
Setting: Seven general hospitals in the Netherlands.
Patients: 152 outpatients with heart failure or patients admitted to hospital because of heart failure, who were included in a randomised trial to assess the impact of a pharmacist led intervention to improve drug compliance. Duration of follow up was at least 18 months.
Main outcome measures: Multivariable logistic regression modelling was used to evaluate information from history, physical examination (for example, blood pressure), drug use, and quality of life questionnaires that independently contributed to the prediction of death. The area under receiver operating characteristic curves (AUC) was used to estimate the predictive ability of the prognostic models.
Results: During the 18 months of follow up, 51 patients (34%) died. Independent predictors of mortality were diabetes mellitus, a history of renal dysfunction (or higher creatinine), New York Heart Association (NYHA) functional class III or IV, lower weight or body mass index, lower blood pressure, ankle oedema, and higher scores on a disease specific quality of life questionnaire. The use of β blockers was predictive of a better prognosis. These factors were used to derive various prediction formulas. A model based on medical history, weight, presence of oedema, and lower blood pressure had an AUC of 0.77. Addition of use of β blockers to this model improved the AUC to 0.80. Addition of NYHA class increased the AUC to 0.84. Data on quality of life did not improve the AUC further (AUC 0.85).
Conclusions: A prognostic model produced on the basis of easily obtainable information from medical history and physical examination can adequately stratify heart failure patients according to their short term risk of death.
- heart failure
- predictive models
- AUC, area under receiver operating characteristic curve
- NYHA, New York Heart Association
- ROC, receiver operating characteristic curve
Statistics from Altmetric.com
- AUC, area under receiver operating characteristic curve
- NYHA, New York Heart Association
- ROC, receiver operating characteristic curve
Mortality among patients with heart failure discharged from hospital has repeatedly been reported to be high.1–4 Determinants of early death or readmission to hospital in these patients have been investigated in several studies. A wide variety of factors is reported to be associated with an increased risk of hospital admission or death, including demographic factors (for example, male sex and single marital status), clinical characteristics (lower systolic blood pressure, renal dysfunction), history of heart failure (previous hospital admissions), and comorbidity (diabetes and depression).2,5–8 Some of these studies focused on invasive and non-invasive test results (such as echocardiographically determined ejection fraction) which are not widely available for all patients, particularly those managed mainly in primary care.5–7
Our aim in this study was to develop a comprehensive and easily applicable prognostic model predicting the risk of death in patients with heart failure, based on information that is readily available in medical practice.
Patients and prognostic determinants
The study group consisted of 152 patients (average age 69.7 years, range 37–91 years) enrolled in a randomised controlled trial evaluating the effect of a pharmacist led intervention on drug compliance in patients with heart failure. All patients in the study were treated with loop diuretics and were either admitted to one of the participating hospitals because of heart failure (ICD-9, 428) or were treated in a specialised outpatient heart failure clinic. Eligible patients with heart failure were included by their cardiologist. The diagnosis of heart failure was validated using the patient’s hospital records, including cardiac imaging findings. Patients with severe psychiatric problems or dementia, those with a planned admission to a nursing home, those who did not manage their own drug treatment (for example, where the treatment was given by relatives or district nurses), and those with a life expectancy of less than three months were excluded from the study.
Patients were enrolled in seven hospitals in the province of Utrecht, Netherlands (one university hospital and six regional hospitals). Most of the patients (70%) were enrolled in two large regional hospitals (containing more than 500 beds). The prognostic model was developed in all 152 patients included in the study.
As potential prognostic determinants, information from the patients’ medical history, physical examination, and laboratory tests was obtained from hospital records, while quality of life was assessed using a generic questionnaire (COOP/WONCA charts) and a disease specific questionnaire (Minnesota “living with heart failure” questionnaire). Information on survival during the 18 months follow up period was retrieved from the hospital records, from the patients’ general practitioners, and from the community pharmacy.
Crude risk ratios for 18 months mortality were calculated for all potential prognostic determinants. Continuous variables were initially analysed without categorisation, but various cut off values were evaluated as well. All factors with a probability value of p < 0.10, together with age and sex, were included in multivariable logistic regression analyses.
Models were constructed in accordance with the chronology in which predictors are available in clinical practice. Hence, we first included all variables from the patient’s history into an overall “history model.” Model reduction was done by excluding variables with p values > 0.10. The reduced history model was consecutively extended with data from physical examination—for example, body mass index, blood pressure variables—and laboratory tests to determine their added value in predicting death. For each model, the reliability (goodness of fit) was quantified using the Hosmer and Lemeshow test. Goodness of fit statistics examine the difference between the observed frequency and the expected frequency for groups of patients. The statistic can be used to determine if the model provides a good fit for the data. If the p value is large, the model is well calibrated and fits the data well.9 The predicted values from the logistic regression model were used to construct receiver operating characteristic (ROC) curves and to calculate the area under the ROC curves (AUC).10 The ROC area is a suitable parameter to summarise the discriminative or predictive value and can range from 0.5 (no discrimination, like a coin flip) to 1.0 (perfect discrimination). Subsequently, to obtain an easily applicable prediction rule, the adjusted regression coefficients of the model were multiplied by a factor 10 and rounded to the nearest integer.
Deleting subjects with a missing value on one of the predictors included in the multivariable model (so called complete case analyses) commonly leads to biased results and definitely to a loss of power.11,12 To decrease bias and increase statistical efficiency, it is better to impute these missing values rather than doing a complete case analyses.11,12 Accordingly, we imputed our missing data using the expectation maximisation method available in SPSS (version 10.0) software. Such imputations are based on the correlation between each variable with missing values and all other variables, as estimated from the set of complete subjects.
Within the 18 months follow up period, 51 (34%) of the 152 heart failure patients died (mortality at six and 12 months, 26 (17%) and 43 (28%), respectively). The cause of death was known in 35 of these patients: they all died from a cardiovascular cause (for example, terminal heart failure, sudden cardiac death). The remaining deaths occurred at home and no documentation of the cause was available. Table 1 shows the results of the crude association of potential prognostic determinants with the 18 months mortality. The strongest predictors were New York Heart Association (NYHA) classification, Minnesota heart failure score, renal dysfunction, and the use or non-use of β blocking agents.
The overall clinical model (that is, the first column of table 2) had an AUC of 0.77. The inclusion of the use of β blockers in this model improved the AUC to 0.80, while the introduction of NYHA class III or IV further improved the AUC to 0.84. The introduction of the Minnesota heart failure score yielded a predictive accuracy with an AUC of 0.85. The fit of all models was good: p values of the Hosmer and Lemeshow statistic ranged from 0.2–0.9.
As the clinical model with information on drug treatment combines data readily available for practising clinicians we transformed this model to a scoring rule: age/17 + 4 for male + 9 for presence of diabetes + 17 for history of renal dysfunction + 10 for presence of ankle oedema + 7 for systolic blood pressure < 110 or diastolic blood pressure < 70 − weight/3 + 13 for absence of use of β blockers (table 3). Such a scoring rule can be considered as one overall measure for predicting mortality in heart failure patients. The score was calculated for each subject by assigning points for each predictor present and adding these points. For instance, a woman of 60 years and weighing 70 kg, with a history of renal insufficiency, diabetes, ankle oedema, and a blood pressure of 130/80 and who does not use a β blocker receives a score of (60/17 + 9 + 17 + 10 − 70/3 +13) = 29.2. In our data, the score ranged from −32.2 to +31.7 (mean −0.3, median 0.5) and the AUC of the rule was 0.80 (95% confidence interval 0.72 to 0.88).
Table 4 shows the incidence of mortality among patients across different score categories. From this table one can directly obtain the observed mortality per score category (reading horizontally). For example, of 23 subjects with a score of > 11, 78% (n = 18) died, while only 12% (n = 3) of the 25 subjects with a score of < −15 died. Similarly, the positive and negative predictive values for various cut off points can be calculated: the positive predictive value for a score ≥ 7 is 31/48 = 65%, while the negative predictive value of a score < 7 is 84/104 = 81%. Reading the table vertically provides estimates of the sensitivity and specificity at different thresholds. For example, 74 (26 + 25 + 23) subjects received a score of ≥ 1. Of these, 43 (12 + 13 + 18) indeed died, correctly predicting 84% of all deaths (that is, the sensitivity or true positive rate). As 31 (31%) of all subjects predicted as future deaths did not die, the specificity of a threshold of 1 was 100 − 31% = 69%.
This study shows that a combination of easily obtainable variables accurately predicts 18 months mortality in patients with heart failure. Except for the use of β blockers, weight, and quality of life score, these predictors were also used in earlier studies (table 5).
Several studies have used multivariate logistic regression to derive predictive models.1,2,5–7,13–15 Most of these studies, however, involved highly selected and mostly relatively young patient populations (for example, patients referred for cardiac transplantation). Our study was undertaken in patients included in a randomised controlled trial. Although there were few inclusion or exclusion criteria in this trial, patients with severe psychiatric problems or dementia, those with a planned admission to a nursing home, those who did not manage their own drug treatment, and those with a life expectancy of less than three months (in the opinion of their physician) were excluded. The exclusion of patients with a short life expectancy mainly led to the exclusion of those with malignancies and other comorbidities, but not to the exclusion of those with moderate or severe heart failure. It should be borne in mind that these patients were in a clinical trial designed to improve compliance. Patients had to give their informed consent and this is likely to have led to the selection of a group of relatively motivated patients. However, it is not plausible that this would have influenced the predictive value of objective parameters such as the presence of diabetes or renal dysfunction.
Other studies have often included variables that are not widely available in heart failure patients. The determinants included in our study, however, are usually routinely available and they turned out to be at least as predictive of mortality as those more specialised variables included in other studies.6,7 Our predictive values were comparable with those in a recent study in which age, the presence of pulmonary crepitations, a lower systolic blood pressure, and higher creatinine concentrations were most predictive of mortality.2 That study, however, did not report the prognostic value of diabetes, which was an important predictor in our study as well as in another study.1
Although results of echocardiography (95%), chest radiography (87%), and electrocardiography (99%) were available in most of our cohort, more specific information on these data, such as ejection fraction (54%) and diastolic function (35%), were only available in a proportion of the participants and were subject to large intrahospital differences. These findings were therefore not included in the analyses.
Finally, although other studies have composed logistic regression models, they did not assess the prognostic performance of a scoring rule combining the individual predictors derived from the model. Although there is no consensus on sample size estimations in studies deriving multivariable prognostic models, as a rule of thumb several investigators recommend 10 or more events per variable to allow a robust estimation of the coefficients.16 As we included more variables in our models, the precision of some of our estimates may be limited. However, limitation of our analyses to the seven determinants most likely to influence prognosis, using the clinical + drug treatment at baseline model excluding age and sex, yielded the same ROC area: 0.80 (95% confidence interval 0.71 to 0.87).
We did not have data on a second cohort of patients with heart failure. Therefore we could not carry out an external validation study. Our scoring rule will be validated in another ongoing study in our group. Ideally, such a validation should take place before the model can be applied in practice.
Easily obtainable clinical data can identify a group of heart failure patients at increased risk of mortality. In their daily practice physicians normally try to estimate the prognosis of their individual patients. Our scoring rule enables them to do this in a more rational way, which should be helpful in decision making. For example, patients with a very poor prognosis might be excluded from invasive treatments. On the other hand, a focus on the management of diabetes, improvement of renal function, attention to cachexia, and optimisation of drug treatment in patients identified as at high risk of early death may be of value. Although quality of life scores are independent predictors of mortality, their added prognostic value is too small to warrant quality of life measurements for such a purpose in routine clinical practice. Additional research is necessary to validate the proposed model in other populations.
We would like to acknowledge Dr Karel G M Moons from the Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, the Netherlands, for his help in developing the statistical model.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.