Objective With the rapid influx of COVID-19 admissions during the first wave of the pandemic, there was an obvious need for an efficient and streamlined risk stratification tool to aid in triaging. To this date, no clinical prediction tool exists for patients presenting to the hospital with COVID-19 infection.
Methods This is a retrospective cohort study of patients admitted in one of 13 Northwell Health Hospitals, located in the wider New York Metropolitan area between 1 March 2020 and 27 April 2020. Inclusion criteria were a positive SARS-CoV-2 nasal swab, a 12-lead ECG within 48 hours, and a complete basic metabolic panel within 96 hours of presentation.
Results All-cause, in-hospital mortality was 27.1% among 7098 patients. Independent predictors of mortality included demographic characteristics (male gender, race and increased age), presenting vitals (oxygen saturation <92% and heart rate >120 bpm), metabolic panel values (serum lactate >2.0 mmol/L, sodium >145, mmol/L, blood urea nitrogen >40 mmol/L, aspartate aminotransferase >40 U/L, Creatinine >1.3 mg/dL and glycose >100 mg/L) and comorbidities (congestive heart failure, chronic obstructive pulmonary disease and coronary artery disease). In addition to those, our analysis showed that delayed cardiac repolarisation (QT corrected for heart rate (QTc) >500 ms) was independently associated with mortality (OR 1.41, 95% CI 1.05 to 1.90). Previously mentioned parameters were incorporated into a risk score that accurately predicted in-hospital mortality (AUC 0.78).
Conclusion In the largest cohort of COVID-19 patients with complete ECG data on presentation, we found that in addition to demographics, presenting vitals, clinical history and basic metabolic panel values, QTc >500 ms is an independent risk factor for in-hospital mortality.
- risk factors
Data availability statement
Data are available on reasonable request.
This article is made freely available for personal use in accordance with BMJ’s website terms and conditions for the duration of the covid-19 pandemic or until otherwise determined by BMJ. You may use, download and print the article for any lawful, non-commercial purpose (including text and data mining) provided that all copyright notices and trade marks are retained.https://bmj.com/coronavirus/usage
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
COVID-19, caused by SARS-CoV-2, was declared a global pandemic by the WHO and has resulted in widespread mortality.1 Cardiac complications and malignant arrythmias,2 potentiated by myocardial injury, have been widely reported in these patients and are associated with both increased in-hospital mortality and out-of-hospital sudden death.3 Several mechanisms have been shown to potentiate arrhythmogenesis in patients with COVID-19 including metabolic and electrolyte disturbances, cytokine storm, hypoxia and certain medications.
A multitude of newly diagnosed ECG changes have been reported in patients with COVID-19 including sinus tachycardia, atrial fibrillation, atrioventricular block, abnormal axis, left bundle branch block (LBBB), right bundle branch block (RBBB), intraventricular conduction delay, QT corrected for heart rate (QTc) interval delay, ST-T changes and Brugada pattern.1 4–9 The use of ECG as a prognostic tool in patients presenting with COVID-19 has been explored in several small studies that have shown abnormalities are associated with increased in-hospital mortality risk.10
The rapid surge of patients with COVID-19 to emergency departments necessitated difficult triaging decisions in the absence of a reliable tool for risk stratification.11 The 12- lead ECG is a non-invasive and widely available tool that has been used as a risk stratification tool in conjunction with standard vital signs and lab values for many diseases; however, its utility in COVID-19 patients has not been explored.11 With this large cohort retrospective study, we sought to identify ECG characteristics that along with clinical and initial laboratory tests are associated with increased in-hospital mortality in patients admitted with COVID-19.
Between 1 March 2020 and 27 April 2020, adult patients who presented to 1 of 13 hospitals within the Northwell Health System with COVID-19 related symptoms were included in the cohort. Northwell Health is the largest healthcare provider in the state of New York, consisting of 24 hospitals. From that cohort, we excluded from our analysis patients with absent or negative SARS-CoV-2 nasal swab, patients who only had an emergency visit encounter without admission and those with discrepant admission data or unknown discharge disposition status. Finally, we analysed only patients with a 12-lead ECG and a complete basic metabolic panel (aspartate aminotransferase (AST), blood urea nitrogen (BUN), creatinine, glucose, potassium and sodium) within the first 48 hours and 96 hours, respectively, from initial presentation).
Data collection and covariates
We included demographic, laboratory, clinical and ECG covariates for our prediction models. Demographic covariates were age, gender, race, ethnicity and insurance type. Presenting vitals that were considered were oxygen saturation (SpO2), mean blood pressure (MBP) and ventricular rate. Laboratory covariates included magnesium, AST, sodium, potassium, BUN, creatinine, lactate and glucose. The medical diagnoses considered for inclusion in our prediction models were hypertension, coronary artery disease (CAD), peripheral vascular disease, chronic obstructive pulmonary disease (COPD), asthma, end-stage renal disease on dialysis, chronic kidney diseases, diabetes and chronic heart failure (CHF). Lastly, ECG covariates were ventricular rate, PR interval, QRS duration, QTc interval and presence of RBBB. ECGs were primarily performed using the MAC 5500 HD System (GE Healthcare, Little Chalfont, Buckinghamshire, UK), which uses automated software (MUSE, GE Healthcare, Little Chalfont, Buckinghamshire, UK) to measure ECG parameters and analyse QTc. All ECGs were manually adjudicated by a cardiologist.
Due to the vast amount of missingness, we did not consider body mass index, C reactive protein, ferritin and troponin. In addition, due to low frequency (<1%), we did not include presence of atrial fibrillation, paced rhythm, q waves, LBBB and ST elevation.
In-hospital mortality status was evaluated through the patients’ hospital admissions (including readmissions, if applicable) during the study period, ending 31 May 2020. Multiple admissions for the same patient within the period were evaluated as one encounter for each respective patient. All-cause mortality was evaluated as further subclassification was not possible. Patients who were still hospitalised as of 31 May or whose in-hospital death status as of 31 May was inconclusive were excluded from analysis. The primary endpoint was in-hospital death at any time during admission.
Covariates were analysed as categorical (binary or multiple categories). For key categorical variables of interest, as specified further, if a subject was missing status, an unknown or not performed category was created, as appropriate. For other variables of interest, only complete cases were used in analysis as outlined via the inclusion/exclusion criteria in figure 1. Baseline demographics were categorised as displayed in table 1.
Presenting vitals were operationalised in binary groups as follows: ventricular rate ≤ versus >120 bpm, MBP ≤ versus >65 mm Hg, SpO2 ≤ versus >92%. Magnesium and lactate had three levels in analysis (normal vs abnormal vs not performed status) as did sodium and potassium (abnormally low vs normal vs abnormally high). Remaining metabolic panel covariates were operationalised as binary factors (normal vs abnormal status). We followed the normal range of laboratory values used by the Northwell core laboratory (table 1). Similarly, ECG characteristics were operationalised in a binary manner as per existing literature as follows: QRS width ≤ versus >120 ms, PR ≤ versus >200 ms, QTc interval ≤ versus >500 ms. QTc interval >500 ms is consistent with cut-off levels used in prior large cohort studies.12 Descriptive statistics were computed for the sample overall and stratified by in-hospital mortality status for the entire cohort, as well as the training and validation cohorts. Baseline patient characteristics were compared according to mortality status using χ2 tests for categorical factors and independent t-tests or Wilcoxon rank sum tests for continuous factors, as appropriate.
To build a prediction model for in-hospital mortality, multivariable logistic regression was used. Relevant risk factors were selected using backward elimination and alpha level of 0.05. The prediction model was based on demographic, vitals, medical history, metabolic panel values at presentation and ECG factors. Multicollinearity was assessed; factors that were multicollinear were assessed in separate models.
To assess the predictive performance (calibration and discrimination) of the developed model, the sample was divided into a training, internal validation and external validation datasets. For both the training and validation datasets, patients from all hospitals were included except for patients from two hospitals consisting of a unique patient population, which were used to create the external validation set. The remaining data were randomly split 70% for training and 30% for validation. The training dataset was used for variable and model selection; the validation dataset was used to assess model performance and internally validate the model. The selected prediction model was then ‘evaluated’ using the external validation dataset.
Internal and external validation methods included discrimination, assessed with area under the receiver operating characteristic curve (Area under curve) and a calibration plot. The Hosmer-Lemeshow goodness of fit (GOF) test was used to compare the expected and observed outcomes. Corresponding 95% CIs around the computed AUC for both the validation and external validation datasets were also calculated in addition to prediction error rates. The model with the greatest AUC was selected for external validation. The Brier score was also calculated as a measure of accuracy. For all analyses, a result yielding p value <0.05 was considered statistically significant. All analyses were conducted using SAS V.9.4 (SAS Institute Inc).
A risk prediction tool was developed that can be easily applied at bedside. Each variable used in the tool were given weighted scores based on the beta coefficients from the model developed using the training cohort. The scoring algorithm to develop weights for each factor involved multiplying each coefficient by 10 and then rounding this value to the nearest integer.12 The ‘points’ for each variable were then summed to get a total risk score for each patient.
Analysis was performed using the cohort of patients with complete baseline data (n=7098, 63.0% of original cohort figure 1). Among all patients who met inclusion criteria, only 2% who were admitted between 1 March and 27 April 2020 did not have a known outcome status as of 31 May 2020. Of the remaining subjects who met all inclusion criteria, 4343 subjects were set aside for the training set, 1861 for the internal validation cohort and 894 for the external validation cohort.
Out of 7098 subjects, 1926 died in-hospital (27.1%). Baseline characteristics for those who were discharged alive and those who died during their hospitalisation are shown in table 1. Almost 65% of males died compared with 35% of females. Almost 18% of subjects aged 85 years or older died in-hospital. Significantly greater proportions of those with abnormal levels of magnesium, AST, glucose, sodium, potassium, BUN or lactate died. Except for asthma, subjects with any of the other evaluated comorbidities were found more often in patients who expired compared with those who did not have these conditions. Finally, a higher percentage of those who were tachycardic and had longer QTc died (table 1).
Comparability of the training, internal validation and external validation cohorts
The in-hospital mortality rates were comparable in the training (26.1%) and internal validation cohorts (26.8%). The in-hospital mortality rate was significantly higher in the external validation cohort (32.7%). When comparing the training and internal validation cohorts, all baseline demographic, electrocardiographic and clinical characteristics demonstrated no significant differences, except for proportion of subjects with history of asthma (9.3% vs 7.7% of subjects in the training and internal validation cohorts).
Final multivariable logistic regression model for in-hospital mortality
The final model included heart rate, QTc, SpO2, sex, race, age, magnesium level, AST level, sodium level, potassium level, BUN level, creatinine level, lactate level, glucose level, medical history of CHF, COPD and CAD. Parameter estimates (beta coefficients) from the model, SEs, ORs and corresponding 95% CIs and p values are shown in table 2.
Strong demographic risk factors associated with in-hospital mortality included male gender (OR: 1.35, 95% CI 1.14 to 1.59) and race. Prior studies have shown a higher rate of men presenting to the hospital with COVID-19, which a higher rate of in-hospital mortality observed.13 Specifically, the odds of in-hospital mortality were significantly lower for African-Americans compared with Caucasians (OR: 0.75, 95% CI 0.61 to 0.94) after covariate adjustment. This finding is consistent with prior reports of lower in hospital mortality for African-Americans, potentially due to elevated rates of out of hospital death.14 The odds of in-hospital mortality for Asians were greater than those for Caucasians (OR: 1.31, 95% CI 0.99 to 1.74).
When compared with those aged 18 to less than 45 years old, subjects 85 years old or older had 6.87 times greater odds of dying in-hospital (95% CI 4.39 to 10.73). Those aged 75 to less than 85 years had 5.17 times greater odds of dying in-hospital compared with patients between the age of 18 to less than 45 years (95% CI 3.42 to 7.82). The ORs (and 95% CIs) of in-hospital mortality for those aged 65– <75 years, 55– <65 years and 45– <55 years compared with those aged 18– <45 years were 3.46 (2.31 to 5.17), 2.44 (1.63 to 3.66) and 1.80 (1.16 to 2.79), respectively.
Presenting vitals associated with greater odds of in-hospital mortality were oxygen saturation <92% and heart rate >120 bpm (OR, 1.79, 95% CI 1.52 to 2.11 and 1.95, 95% CI 1.38 to 2.75, respectively). Metabolic panel data found to be significantly associated with increased odds of in-hospital mortality included abnormally high magnesium, potassium and serum lactate levels along with, hypernatraemia, BUN >23 mg/dL, AST >40 U/L, serum creatinine >1.3 mg/dL, and hyperglycaemic. History of CHF (OR 1.75, 95% CI 1.34 to 2.30), COPD (OR 1.60, 95% CI 1.18 to 2.18) and CAD (OR 1.21, 95% CI 1.02 to 1.44) were each associated with increased odds of in-hospital mortality. Lastly, using all 12-lead ECG measurements and diagnoses, the presence of QTc >500 ms within the first 48 hours of presentation was significantly associated with increased odds of in-hospital mortality (OR: 1.41, 95% CI 1.05 to 1.90).
The discriminative ability of the model (as assessed using AUC) for the training and validation cohorts were 0.79 (95% CI 0.78 to 0.81) (online supplemental eFigure 2) and 0.76 (95% CI 0.74 to 0.79), respectively. The AUC for the external validation cohort was 0.75 (95% CI 0.72 to 0.79) with a predicted error rate of 26%. The model also exhibited almost perfect calibration (online supplemental eFigure 1).
The Brier score for the internally validated and externally validated models were <0.18, providing enough evidence of near excellent accuracy. The results from the Hosmer and Lemeshow GOF tests were not significant, providing insufficient evidence to demonstrate a poor fit of the model using the risk prediction tool (see online supplemental material for details). In summary, the model performs well and provides near accurate prediction for the risk of mortality as evidenced by the low Brier score and non-significant Hosmer-Lemeshow GOF test result.
Risk prediction tool
The range of the risk index score is 0–109. The raw total for the sum score ranged from −12 to 97. As such, a constant of 12 was added to each subject’s sum score to achieve a range from 0 to 109 for easier interpretation. Figure 2 shows the scoring system and corresponding predicted probability of in-hospital mortality in a patient with COVID-19. The relationship between scoring system and mortality follows an S shaped curve. To illustrate, there is a less than 1% predicted mortality for scores less than 13, a less than 5% mortality predicted for scores of less than 30, <30% for scores <51 and predicted mortality of >90% for scores of >81. Figure 3 provides a predicted by observed bar graphic, stratifying the sample into 4 strata of the risk score (low, low–medium, medium–high, high).
Risk stratification on presentation is of particular importance for acute multisystemic clinical syndromes, including COVID-19. Well-described predictors of mortality in the critical care literature have included advanced age, hypotension on presentation, presence of chronic cardiovascular diseases (coronary artery disease, hypertension and chronic heart failure), diabetes mellitus and abnormal metabolic profile.11 These variables appear to also have a prognostic significance but in smaller cohorts of patients with COVID-19.1–3 7–9 While these tools have focused primarily on the respiratory symptoms of COVID-19, a number of cardiac manifestations are well reported in patients with COVID-19 and known to be associated with poor outcomes.3 15 We present here the largest analysis published on patients with COVID-19 and provided additional evidence on the importance of patients’ demographics (male gender, older age and race), presenting vitals (tachycardia and hypoxaemia), basic metabolic panel values and comorbidities (CHF, COPD and CAD) in risk stratifying COVID-19 patients. In addition, we examined the additive prognostic value of the 12-lead ECG and found that QTc >500 ms had an almost twofold increase in the odds of mortality when controlled for multiple demographics, clinical and laboratory covariates.
ECG abnormalities in COVID-19 patients including Brugada-like pattern, transient ST segment elevations, sinus tachycardia, atrial fibrillation, ventricular tachycardia, sinus bradycardia, varying degrees of atrioventricular block, QT interval prolongation, bundle branch block and QRS axis deviation16 have been noted in various case reports and small cohort studies.17 Available studies are greatly limited by the small population size and lack of controlling for known confounders and most notably underlying cardiac disease. In addition, the majority of the described ECG findings have a very low incidence in the overall COVID-19 population, appear in patients during the advanced stage of COVID-19 disease and therefore have a limited role as screening markers. We evaluated available 12-lead ECG measurements and diagnoses for all presenting confirmed COVID-19 patients, and after excluding ECG diagnoses with incidence of <1% and adjusting for known confounders, only QTc interval >500 ms was found to be a significant independent predictor of in-hospital mortality. We previously published in a cohort study of 9564 patients that occurrence of atrial fibrillation during hospitalisation for COVID-19 is an independent predictor of in-hospital mortality.15 The much higher occurrence of atrial fibrillation is because we used both 12-lead ECGs, as well as full disclosure telemetry throughout the entire course of hospitalisation, as opposed to only the initial 12-lead ECG in this analysis.
Cardiac depolarisation and repolarisation are complex and delicate processes that can be affected by cardiac dysfunction and by metabolic and electrolyte imbalances as well as medications, factors that are all affected in patients with COVID-19. QTc prolongation is also a marker for systemic illness severity, increased mortality and an independent risk factor for sudden death both in the general population and intensive care unit. Yu et al 12 in a series of 41 096 patients found that QTc >500 was a significant predictor of all-cause mortality. The FROG-ICU study, a prospective observational study conducted throughout 21 ICUs in France and Belgium, found QTc was prolonged in 14% of 1467 patients.18 After adjusting for confounders, the study also found prolonged QTc to be associated with risk of 30-day death (HR 1.55) and 1 year death (HR 1.31).19 SARS-CoV-2 is known to have a high affinity for the ACE2 receptor resulting in an overexpression of angiotensin-II. Angiotensin-II (ACE2) inhibits potassium channels in myocytes leading to excess sodium influx or reduced potassium efflux that results in prolonged ventricular repolarisation and subsequent prolonged QTc.20–24 Prolonged ventricular repolarisation in conjunction with an early after depolarisation trigger can result in lethal arrythmias including Torsade de pointes.25 SARS-CoV-2 is also thought to cause cardiac injury through overexpression of ACE2 resulting in microvascular injury.26 27 A study by Chen et al 7 found prolonged QTc to an independent predictor for cardiac injury in a cohort of 63 patients. Presence of cardiac injury results in a higher mortality rate in patients with COVID-19,28 29 and QTc prolongation on an ECG can be used as a valuable prognostic tool.29 This analysis focused on the admission ECG to eliminate the possibility of pharmaceutical exposure to QT-prolonging drugs commonly used as therapeutics for COVID-19 in the in-patient setting or prescribed in the outpatient setting.
Furthermore, we incorporated clinical information readily available or easily obtainable on presentation into a point-based risk prediction score. The main objective of our paper is to use parameters that are readily available in most clinical settings and come up with a prediction model with broader applicability. For this reason, we elected to include data with labs not performed as a category for test results. The reasoning is certainly multifactorial and included differences in common practice based on the type of facility where the patient initially presented, degree of acuity and patient volume, but also due to the unpredictable rapid disease progression in certain patients. In general, we believe that the availability of certain data is correlated with severity of disease, which is likely to be correlated with survival. Our model demonstrates a strong predictive accuracy for in-hospital mortality (AUC 0.78) using a scoring system with a scale 0–109 points. We believe our scoring system can be incorporated into triaging workflows to help identify high-risk patients on presentation.
This retrospective study evaluated a large cohort for ECG and clinical characteristics that were predictive of increased in-hospital mortality while adjusting for potential and known confounders. While we present these factors as a useful tool for risk stratification, a randomised prospective trial would need to be conducted to fully evaluate its effectiveness and validity. The rapid surge of the first COVID-19 wave in New York negatively affected clinical documentation and limited testing. This practice could have introduced bias in patient selection and errors in identification of comorbidities. In addition, it affected the degree of laboratory missingness in our cohort. However, missing values participated in the model as ‘not performed’ to minimise selection bias. We elected to include data with labs not performed as a category for test results, necessitating categorising test results into categories such as normal and abnormal. While categorisation of a continuous variable can be problematic due to loss of information, it can also be pragmatic by creating a categorical variable that is easier to use and interpret in the clinical setting. In such cases, missing becomes an actual value of the laboratory result, which represents the normal course of care, including situations when a test was not ordered. Similarly, we used all-cause in-hospital mortality as outcome as the subclassification of mortality was not possible due to poor documentation in midst of a pandemic.
In the largest cohort of COVID-19 patients with ECG parameters collected at presentation to date, we found that prolonged cardiac repolarisation (QTc interval >500 ms) is an independent predictor of in-hospital mortality along with previously described demographic, clinical and basic metabolic panel information. Our simple risk prediction score based on readily available information on presentation could serve as effective tool for risk stratification in the emergency department to optimise triaging workflows for patients with COVID-19.
What is already known on this subject?
Various clinical prediction tools have been developed to risk stratify patients presenting to the hospital with COVID-19 infection. These tools include a multitude of different variables and are validated in limited patient cohorts.
What might this study add?
Presence of QTc >500 ms on presenting ECG is an independent predictor of in-hospital mortality for patients with COVID-19. The 12-lead ECG along with presenting vital signs, clinical profile and basic metabolic panel values can be used together in a scoring system predicting in-hospital mortality.
How might this impact on clinical practice?
Delayed myocardial cell repolarisation, represented by prolonged QTc, represents a sensitive indicator of myocardial dysfunction due to cardiac and extracardiac factors in patients with COVID-19.
Data availability statement
Data are available on reasonable request.
Patient consent for publication
The study was approved by the Northwell Institutional Review Board. It was not appropriate or possible to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of our research.
JF and KMC are joint first authors.
Twitter @kcole_12, @amarbir88, @NikhilDSharmaMD, @stefzafeirop, @keepinrhythm
JF and KMC contributed equally.
Contributors SEM and JF conceptualised and designed the study. SEM and JF had full access to all data in the study and take responsibility for the integrity ofthe data and the accuracy of the data analysis. JF performed data cleaning, model design, development, training, optimisation and validation. KMC, AB, and NS wrote the draft of the manuscript. JF, SEM and KC designed and created all figures. YL, SZ, TB and UA critically reviewed the paper and provided advice over a period of several months. All authors approved the final submitted research manuscript and agree to be personally accountable for their contribution and for the academic integrity of the work. SEM accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.