Article Text
PDF
Abstract
Objective To develop and validate a calculator to predict the risk of in-hospital mortality in patients with active infective endocarditis (IE) undergoing cardiac surgery.
Methods Thousand two hundred and ninety-nine consecutive patients with IE were prospectively recruited (1996–2014) and retrospectively analysed. Left-sided patients who underwent cardiac surgery (n=671) form our study population and were randomised into development (n=424) and validation (n=247) samples. Variables statistically significant to predict in-mortality were integrated in a multivariable prediction model, the Risk-Endocarditis Score (RISK-E). The predictive performance of the score and four existing surgical scores (European System for Cardiac Operative Risk Evaluation (EuroSCORE) I and II), Prosthesis, Age ≥70, Large Intracardiac Destruction, Staphylococcus, Urgent Surgery, Sex (Female) (PALSUSE), EuroSCORE ≥10) and Society of Thoracic Surgeons’s Infective endocarditis score (STS-IE)) were assessed and compared in our cohort. Finally, an external validation of the RISK-E in a separate population was done.
Results Variables included in the final model were age, prosthetic infection, periannular complications, Staphylococcus aureus or fungi infection, acute renal failure, septic shock, cardiogenic shock and thrombocytopaenia. Area under the receiver operating characteristic curve in the validation sample was 0.82 (95% CI 0.75 to 0.88). The accuracy of the other surgical scores when compared with the RISK-E was inferior (p=0.010). Our score also obtained a good predictive performance, area under the curve 0.76 (95% CI 0.64 to 0.88), in the external validation.
Conclusions IE-specific factors (microorganisms, periannular complications and sepsis) beside classical variables in heart surgery (age, haemodynamic condition and renal failure) independently predicted perioperative mortality in IE. The RISK-E had better ability to predict surgical mortality in patients with IE when compared with other surgical scores.
- Cardiac surgery
- Endocarditis
Statistics from Altmetric.com
Introduction
Nowadays, cardiac surgery is required in more than 50% of patients with active infective endocarditis (IE) and this percentage is even higher in left-sided prosthetic valve infections.1–3
Risk stratification plays an important role in the decision-making for surgery in IE, given the heterogeneous and not uncommonly fatal outcome of these patients.4 A prognostic scoring system, if accurate, could be of help in this scenario. Unfortunately, scoring systems extensively used in heart surgery, such as the European System for Cardiac Operative Risk Evaluation (EuroSCORE) and the Society of Thoracic Surgeons (STS) score, are neither specific nor accurate for IE.5–7
The purposes of this study were (1) to identify independent risk factors for operative mortality in patients with active, left-sided IE; (2) to develop and validate an accurate surgical risk scoring system for clinical decision-making; (3) to compare the predictive performance of our score with that of other cardiac surgery scores5 6 8 9 and (4) to validate the accuracy of our score in an external cohort of patients.
Methods
Study design and patient population
This study was performed at three tertiary care centres in Spain with surgical facilities that have been working together on IE since 1996. Standardised protocols, uniform data collection and identical diagnostic and therapeutic criteria were used. From January 1996 to August 2014, 1299 consecutive patients with IE were prospectively recruited on an ongoing multipurpose database. For this study, all patients with definite, left-sided IE in the active phase of the disease (ie, receiving intravenous antibiotics) who underwent surgery (n=671) were retrospectively analysed.
Patients with right-sided IE (n=156), left-sided IE without surgical indications (n=375) and those with a surgical indication that were not surgical candidates (n=97) were excluded from the study.
Duke criteria were applied until 2002, and modified Duke criteria were applied thereafter.1 This study was approved by the local ethical committees, and informed consent was obtained from each patient.
Preoperative characteristics were entered in a standardised case report form that was defined elsewhere.10
Definition of terms
Nosocomial and community-acquired IE were defined according to the guidelines.1 Persistent signs of infection were defined as persistent bacteraemia or fever after 7 days of appropriate antibiotic treatment, once other possible foci of infection are ruled out.1 Septic shock was defined as acute circulatory failure in sepsis, with persistent systolic pressure <90 mm Hg despite adequate volume resuscitation.10 Cardiogenic shock was considered as systolic pressure <90 mm Hg and tissue hypoperfusion due to myocardial dysfunction, despite adequate preload, and accompanied by low cardiac index and high pulmonary wedge pressure.11 Thrombocytopaenia was defined as a platelet count <1 50 000/mL.12 Renal failure was defined as glomerular filtration rate <60 mL/min/1.73 m2. The diagnosis of systemic embolism was based on clinical signs and data derived from imaging procedures.
Vegetations were measured by transoesophageal echocardiography in various planes, and the maximal diameter was used for subsequent analysis. Perivalvular complications were defined in detail elsewhere.13
All patients underwent surgery before antibiotic regimen was completed and following guidelines recommendations.1 Urgent surgery was defined as surgery done within 2–3 days once clinically decided.
Logistic EuroSCORE I and EuroSCOREII were prospectively recorded since 2000 and 2011, respectively, to assess the operative risk.5 6 For those episodes recruited before these dates, these surgical scores were retrospectively calculated. The prosthetic valve, age ≥70, large intracardiac destruction, Staphylococcus spp, urgent surgery, sex (female), EuroSCORE ≥10 (PALSUSE) and the Society of Thoracic Surgeon’s Infective Endocarditis (STS-IE) risk scores have also been calculated retrospectively.8 9
Primary endpoint was in-hospital mortality after valve surgery, and it was defined as death occurring after surgery and before discharge, regardless of its cause.
Statistical analysis
Construction of the model
To construct the prognostic model, the total cohort was randomly divided (computer generated) into the development sample (two-thirds) and the validation sample (one-third) (figure 1). Clinical, epidemiological, microbiological and echocardiographic characteristics of both samples were compared. Categorical variables are expressed as a frequency and a percentage and were compared with the χ² test and Fisher’s exact test when appropriate. Continuous variables are reported as mean value and SD or median and IQR and were compared by a two-tailed Student’s t-test. In the case of multiple categories, the analysis of variance test was used.
Flow chart. Flow chart showing patient selection and randomisation to create the development and validation samples. IE, infective endocarditis.
The prognostic model was created in the development sample and assessed in the validation sample. The influence of different variables in postoperative mortality was first tested in a univariable analysis. Variables with p values <0.10 and those considered clinically relevant were included in a multivariable logistic regression analysis. The final model was built by means of stepwise forward selection and backward elimination technique. The significance levels for selection and elimination were <0.05 and ≥0.10, respectively. To control for confounding, we compared the estimated parameters of the full model with those of the final selected model. No difference between the estimated parameters exceeded 10%.
Performance of the final predictive model, in both development and validation samples, to predict postoperative death was assessed by analysing discrimination (receiver operating characteristic (ROC) curve), calibration (agreement between predicted and observed probabilities, dividing the sample into deciles of risk) and goodness of fit (Hosmer-Lemeshow test). No significant multicollinearity (assessed using variance inflation factors) was detected.
The weightings of the eight variables included in the final logistic regression model are obtained by multiplying the beta coefficients of each variable by a factor of 10, and then rounding them to the nearest integer to be transformed into risk points (eg, for septic shock, the beta coefficient was 0.702: 0.702×10 = 7.02 ≈ 7 points; for cardiogenic shock, the beta coefficient was 1.486: 1.486×10 = 14.86 ≈ 15).
The Risk-Endocarditis score (RISK-E score) of each patient is then calculated by adding up the points obtained according to the presence of each risk factor. Therefore, in a patient who has none of the variables included in the score, the risk score is 0, and in a patient who has the eight variables present, the total score is 68.
Finally, the predicted probabilities of in-hospital mortality for each combination of points and possible RISK-E score were estimated creating all possible combinations of the eight predictors (seven binary predictors and one with four possible categories) with 512 fictional observations. Then, we performed a univariable logistic regression analysis for in-hospital mortality, with the RISK-E as the independent variable. Subsequently, we obtained the estimated probability calculated by the logistic regression analysis of having the event (in-hospital mortality) for each possible scoring.
Comparison with existing scores
We compared the predictive performance of the RISK-E score to that of four existing surgical risk scores (EuroSCORE I, EuroSCORE II, STS-IE and PALSUSE) in our entire patients’ cohort. Differences in the discriminative power among scores were assessed by comparing their ROC curves.
External validation
We also evaluated the accuracy of the RISK-E score in an external cohort of patients (n=204) with left-sided IE who underwent surgery during the active phase of the disease, from 2010 to 2015 in a tertiary care referral centre for endocarditis in France. ROC curves were used to assess discrimination.
All tests were two sided, and differences were considered statistically significant at p values <0.05. Statistical analysis was performed with Stata V.12.0.
Results
Baseline characteristics
Of 1299 patients with active IE, 1143 were left-sided IE episodes and 671 of them (58.7%) underwent surgery. Mean age of our surgical cohort was 61 (14) years and 69% were men. Forty per cent had prosthetic valve infections, and 31.6% had periannular complications. Our patients were particularly sick: 33.5% had acute renal failure, 10% were in septic shock and 19.4% suffered a central nervous system embolism prior to surgery. Median time from diagnosis to surgery was 9 days (IQR 2–20) and 28.5% underwent surgery within the first 48 hours after diagnosis. Staphylococci were the most common microorganisms (33%). In-hospital mortality occurred in 192 patients (28.6%).
The total surgical cohort was randomised into the development sample (n=424) and the validation sample (n=247). The two samples were similar with respect to the main clinical, microbiological and echocardiographic characteristics (see online Supplementary table S1, S2).
Univariable analysis for in-hospital mortality
In the development sample, patients who died during hospitalisation were older and had more frequently a nosocomial infection. Patients with chronic anaemia and chronic obstructive pulmonary disease died more frequently during hospitalisation. Regarding microorganisms, fungal infections were more frequent in this group. Prosthetic endocarditis, vegetation detection and the presence of periannular complications were associated with in-hospital mortality (table 1).
Epidemiological, clinical, microbiological and echocardiographic characteristics in the derivation cohort
Concerning clinical evolution prior to surgery, preoperative acute renal failure, heart failure, septic shock and thrombocytopaenia were higher in those patients who died during hospitalisation. In these patients, surgery was more frequently performed in an urgent manner (table 1).
Prediction model: creation and internal validation
Variables significantly associated with in-hospital mortality in the univariable analysis and those considered clinically relevant were included in a multivariable logistic regression analysis. Those variables independently associated with in-hospital mortality in the derivation sample were age, prosthetic infection, periannular complications, Staphylococcus aureus and fungi infections, acute renal failure, septic shock, cardiogenic shock and thrombocytopaenia.
These eight variables were included in our final prediction score, the RISK-E score (table 2).
Independent preoperative predictors of mortality in infective endocarditis by logistic regression analysis and the deriving scoring system for mortality prediction after cardiac surgery (RISK-E score)
The score showed a good discrimination, with an area under the ROC curve of 0.77 (95% CI 0.71 to 0.82). The good predictive accuracy of the score was retained when tested in the validation sample, obtaining an area under the ROC curve of 0.82 (95% CI 0.75 to 0.88) (figure 2). In both samples, an excellent correlation between the predicted and observed in-hospital mortality was documented (figure 3). The Hosmer-Lemeshow goodness-of-fit was satisfactory in both samples (p=0.30 in the derivation and p=0.29 in the validation cohort).
Discrimination of the model in development and validation samples. Receiver-operator characteristic (ROC) curves for our logistic regression model and the derived score in the development and validation samples. AUC, area under curve.
Comparison of observed and predicted risk in the entire cohort (calibration). Comparison of observed and predicted mortality in the complete cohort to assess calibration. The sample is divided into deciles of risk. The red line represents the line of identity.
To assess the performance of the score during the study period, discrimination was also evaluated dividing the entire cohort in two different periods: episodes from 1996 to 2004 and those from 2005 to 2014. Sensitivity and specificity of the RISK-E score were good in both cases. The area under the ROC curve for the first and second periods were 0.76 (95% CI 0.67 to 0.83) and 0.84 (95% CI 0.79 to 0.88), respectively.
RISK-E score calculation
To calculate the RISK-E score of each patient, we have to add up the points obtained according to the presence of each risk predictor. The minimum total RISK-E score is 0 for a patient without any risk factors, and the maximum possible score is 68, and the predicted probability of postoperative mortality ranged from 3% for a patient with a score of 0 to 97% for a patient with the highest possible score of 68 (figure 4, see online Supplementary table S3). Table 3 shows how to calculate a patient’s RISK-E score.
Calculation of the RISK-E score in a particular patient
Predicted risk of postoperative mortality associated with individual RISK-E scores. Predicted risk of postoperative mortality associated with individual RISK-E scores, according to the presence and scoring of each risk factor.
RISK-E score validation in patients with surgical indications who did not undergo surgery
Ninety-four patients with surgical indications did not undergo surgery, most of them because of prohibitive surgical risk. The characteristics of these patients are shown in online Supplementary table S4. In this high-surgical risk population, the performance of RISK-E score was very good: area under the ROC curve 0.87 (95% CI 0.79 to 0.94).
Comparison between different surgical risk scores
The discriminative performance of different published surgical risk scores was estimated in our surgical cohort (n=671). The area under the ROC curve for the logistic EuroSCORE was 0.76 (95% CI 0.71 to 0.82), for EuroSCORE II was 0.76 (95% CI 0.70 to 0.82), for STS-IE score was 0.74 (95% CI 0.68 to 0.79) and for PALSUSE score was 0.64 (95% CI 0.58 to 0.68).
A comparison of the areas under the ROC curves showed a statistically significant superior predictive performance of our score (p=0.010), when compared with the others (figure 5).
Comparison of the discriminative performance of different surgical risk scores. Analysis of the discriminative power between logistic EuroSCORE, STS-IE, PALSUSE score and our score (RISK-E) assessed by comparison of receiver-operator characteristic (ROC) curves in 671 patients. (The discriminative power of logistic EuroSCORE and EuroSCORE II in our cohort was identical. Thus, EuroSCORE II is not represented in the graphic to avoid confusion). AUC, area under the curve; EuroSCORE, European System for Cardiac Operative Risk Evaluation; PALSUSE, Prosthesis, Age ≥70, Large Intracardiac Destruction, Staphylococcus, Urgent Surgery, Sex (female); STS-IE, Society of Thoracic Surgeons’s Infective Endocarditis Score.
External validation
Mean age of the external cohort (n=204) was 60 (15) years, 32.8% of patients had prosthetic valve endocarditis, 5.4% had cardiogenic shock, 24% had acute renal insufficiency, 46.6% had periannular complications and 2.9% had septic shock. In-hospital mortality was 8.8%.
In this cohort, the RISK-E score also obtained a good predictive performance. The area under the ROC curve to predict postoperative mortality was 0.76 (95% CI 0.64 to 0.88).
Discussion
This study is the largest prospective cohort of patients with left-sided IE who underwent surgery during the active phase of the disease, in which independent prognostic variables for perioperative mortality have been analysed. Our study has developed and validated a simple bedside prediction score (RISK-E Score) that can be used to calculate the risk of patients with active, left-sided IE who need surgery as an aid to clinical decision-making in routine practice. The predictive accuracy of this novel surgical risk score performed better relative to four other surgical risk scores, and the internal and external validation guarantees its applicability.
Our model includes both predictors related to patient’s characteristics classically associated to increased risk in cardiac surgery and well represented in general scores5 6 14 and others exclusively associated with the disease itself.1 4 The eightvariables included in the final model (age, prosthetic infection, periannular complications, S. aureus or fungi infection, acute renal failure, septic shock, cardiogenic shock and thrombocytopaenia) are clinically relevant and easy to obtain, so the score can be calculated at any time during the clinical evolution of a particular patient.
Why a new surgical risk score?
The scores currently used in clinical practice do not represent patients with IE and the few specifically created for IE have many limitations. EuroSCORE I, EuroSCORE II and STS score, although widely used in cardiac surgery, are not specific for IE,5 6 14 and they are far from being accurate. The performance of logistic EuroSCORE I and EuroSCORE II has been previously assessed in patients with IE with contradictory results.7 15–17 The most recent study showed that both significantly overestimated operative mortality.7 However, because the representation of cases with active IE in these scores is minimal, they should be used with caution. Thus, significant efforts have been made to develop a surgical risk score specifically devoted to patients with IE.
The largest surgical score focused on patients with IE was done by Gaca et al.9 They used 19 543 surgical procedures from the STS database. The authors described a model with 13 variables to help in clinical decision-making. Unfortunately, there are a number of issues that prevent their applicability: only 51.5% of the patients had active IE; in fact 42.95% of operations were elective, microbiological information was not provided, prosthetic and native valves were analysed together and anatomic factors, such as periannular complications, were not considered, all of which should be of paramount importance when considering surgical outcome.
De Feo et al also developed a risk score in their single-centre study of 440 native valve IE patients undergoing surgery.17 Six predictors were identified: age, renal failure, New York Heart Association (NYHA) class IV, critical perioperative state, lack of preoperative attainment of blood culture negativity and perivalvular involvement. This score is relatively simple, and it seems to be not inferior to the STS-IE score.9 However, it was derived exclusively from native valves, so it is not applicable to prosthetic IE, 17% of cases were healed IE, right-sided infections were included and most episodes were due to streptococci. In consequence, mortality was low (9.1%).
Finally, Martínez-Sellés et al 8 recently published a surgical risk score (PALSUSE) from a multicentre cohort of patients with IE. The score was developed using seven variables: prosthetic valve, age ≥70, large intracardiac destruction, Staphylococcus infection, urgent surgery, sex (female) and logistic EuroSCORE ≥10. The main limitations are methodological: no internal validation was performed, and several variables incorporated as independent predictors (age, sex and urgent surgery) are already integrated in the logistic EuroSCORE. This may lead to significant multicollinearity. In addition, 11% of cases were isolated cardiac device-related IE.
The reasoning behind the RISK-E Score
Most variables included in our model are consistent with those identified in previous scores: age, prosthetic valve IE, substantial intracardiac destruction, infection due to virulent microorganisms, acute renal failure and cardiogenic shock.5 6 8 9 17 The two variables (septic shock and thrombocytopaenia) that had not been previously associated with perioperative mortality have already been described as prognostic factors related to poor outcome in patients with IE.10 12
Cardiogenic shock was found to be the most powerful predictor of postoperative mortality. This is in agreement with the score published by Gaca et al 9 and other risk models for cardiac surgery.18 Undoubtedly, patients who face intervention in this extreme haemodynamic condition have a high-risk surgery. However, this situation should not discourage surgery, as it has been demonstrated that prompt intervention in patients with left-sided IE and cardiogenic shock provides satisfactory results.19 20
In our study, age and renal failure were also important independent predictors of postoperative mortality. They are a constant in most cardiac surgery scores, strongly correlating with morbidity and mortality.5 6 8 9 14 17 21 Renal failure has been associated with poor outcome in any sort of cardiac surgery, especially in IE.17 22
Prosthetic valve endocarditis and its prognostic implications are well known.2 23–25 It is associated with a high incidence of periannular complications,15 25 which represents not only a marker of a more virulent infection but also a more complex cardiac surgery, usually requiring challenging reconstruction techniques and longer surgery times.1 2 8 Taken together, these factors lead to a higher need for cardiac surgery and explain why prosthetic IE is associated with some of the highest mortality rates observed in bacterial infections.1 24 25
Regarding infectious factors, one of the most important variables of our model is infection due to virulent microorganisms. S. aureus and fungi have been extensively recognised as independent predictors of mortality.4 26 27 Interestingly, except in PALSUSE score,8 microorganisms are not represented in any other surgical risk score.
Septic shock and thrombocytopaenia are score’s representative variables of a particular fearsome aspect of IE: uncontrolled infection. The ominous prognostic impact on in-hospital mortality of these factors in IE has already been shown.10 12 28 A synergistic interaction between thrombocytopaenia and S. aureus bacteraemia on mortality has also been documented.12 29 According to some authors,19 septic shock in IE has a worse prognosis than cardiogenic shock. However, in our scoring system, septic shock has a much lower value than cardiogenic shock. One fact might explain this seemingly paradoxical result: in the whole series, only 44% of patients with septic shock underwent surgery, most probably those who were less sick.
In our model, preoperative stroke was not a predictor of postoperative mortality. Neurological complications in IE have been associated with higher in-hospital mortality. However, García-Cabrera et al 30 found that only moderate-to-severe ischaemic events and cerebral haemorrhages were significantly associated with a poorer outcome. It is also known that cardiac surgery is conditioned by the presence of these complications.1 In fact, in this work, patients with neurological complications and a formal surgical indication underwent surgery less frequently than those without these complications.30 Taken together, the heterogeneity of the severity of neurological complications and a lower frequency of surgery in this group of patients may explain why neurological complications in our study were not associated with postoperative mortality.
Why urgent status was not independently associated with postoperative mortality in our score makes sense; two critical variables already included in the model (cardiogenic and septic shock) are the main indications for emergency or urgent surgery. Thus, patients who need emergency surgery are already represented in our score.
Is RISK-E Score relevant in clinical practice?
In clinical practice, RISK-E score is clearly superior to the other existing scores in a significant proportion of patients. Just two examples to illustrate this situation. Case one: a 35-year-old woman with mitral valve prolapse, severe mitral regurgitation, viridans group streptococci bacteraemia and a lower limb arterial embolism has a EuroSCORE II of 1.75% and a RISK-E score of 0, which corresponds to a predicted mortality of 3%. Case two: a 60-year-old man, with prosthetic mitral and aortic IE due to S. aureus, with periprosthetic aortic abscess, acute renal failure, thrombocytopaenia and septic shock has a EuroSCORE II of 23%, whereas the RISK-E score is 48, with a predicted mortality of 81%.
In the first case, both scores performed similar. On the contrary, in the second case, where infectious factors have a prominent role, RISK-E score performed much more accurately, reflecting the patient’s real risk.
Limitations
This study has several limitations. First, it is part of a multiproposal prospective collection of data, with a large number of cases, retrospectively analysed, so it has the potential bias inherent of observational studies. It may also have a potentially referral bias because all the participants are tertiary care centres. The reason to perform an external validation was to increase the generalisability of our results, but since the external cohort was also a referral centre, the results of the present work should be mainly applied to large reference centres with the same characteristics. Second, the STS score was not calculated in our cohort. A retrospective estimation of this score was not possible due to the lack of relevant data. Third, although our database collects detailed clinical, microbiological and echocardiographic data on the majority of known risk factors for operative mortality, it is possible that other important risk factors exist (ie, frailty) and were not captured in our data set.
Conclusions
The RISK-E Score is a novel, user-friendly score to estimate the postoperative mortality among patients with active left-sided IE, which has been both internally and externally validated. We found that adding IE-specific predictors of mortality (virulent microorganisms, periannular complications and sepsis manifestations) to the well-established classical factors (age, haemodynamic conditions and renal failure), our model had a superior predictive accuracy than the other available surgical risk scores.
Thus, this risk-scoring model should be a useful tool for clinical decision-making, providing more accurate information to patients and their families.
Key questions
What is already known on this subject?
Risk stratification plays a key role in the decision-making for cardiac surgery. Unfortunately, scoring systems used in heart surgery are neither specific nor accurate for patients with infective endocarditis.
What might this study add?
Taking into account specific infectious factors, as well as classical surgical factors, we have created a score, the Risk-Endocarditis Score (RISK-E). The eight variables included in the score are age (≤51: 0; 52-63: 9; 64-72: 13; ≥73 years old: 14 points), prosthetic infection (six points), periannular complications (five points), Staphylococcus aureus and fungi infections (nine points), acute renal failure (five points), septic shock (seven points), cardiogenic shock (15 points) and thrombocytopaenia (seven points). The RISK-E score of a patient is calculated by adding up the points obtained according to the presence of each risk factor.
The herein score has better ability to predict surgical mortality in patients with active left-sided endocarditis undergoing surgery than other available surgical risk scores.
How might this impact on clinical practice?
Considering the superior accuracy of our score, it should be a useful tool for clinical decision-making in patients with active left-sided endocarditis and surgical indications, providing more precise information to clinicians and patients.
References
Footnotes
Contributors CO, IV, ASR, CS, CFP and JL had the original idea and wrote the initial draft of the manuscript. CO and CFP had full access to all the data in the study and took responsibility for the integrity of the data and the accuracy of the data analysis. CF, ES, SH, GH, GT, AFF, CS, CSE, EGG, BO, AG, MC, LM, JC, DV, SDS and JB contributed substantially to the study design, data analysis and interpretation and the writing of the manuscript.
Competing interests None declared.
Ethics approval Hospital Clnico San Carlos ethical committee.
Provenance and peer review Not commissioned; externally peer reviewed.