Report of the STS Quality Measurement Task Force

Quality Measurement in Adult Cardiac Surgery: Part 1—Conceptual Framework and Measure Selection

As diagnostic and prognostic models developed by traditional statistics perform poorly in real-world, artificial intelligence (AI) and Big Data (BD) may improve the supply chain of heart transplantation (HTx), allocation opportunities, correct treatments, and finally optimize HTx outcome. We explored available studies, and discussed opportunities and limits of medical application of AI to the field of HTx.

A systematic overview of studies published up to December 31st, 2022, in English on peer-revied journals, have been identified through PUBMED-MEDLINE-WEB of Science, referring to HTx, AI, BD. Studies were grouped in 4 domains based on main studies’ objectives and results: etiology, diagnosis, prognosis, treatment. A systematic attempt was made to evaluate studies by the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD).

Among the 27 publications selected, none used AI applied to BD. Of the selected studies, 4 fell in the domain of etiology, 6 in the domain of diagnosis, 3 in the domain of treatment, and 17 in that of prognosis, as AI was most frequently used for algorithmic prediction and discrimination of survival, but in retrospective cohorts and registries. AI-based algorithms appeared superior to probabilistic functions to predict patterns, but external validation was rarely employed. Indeed, based on PROBAST, selected studies showed, to some extent, significant risk of bias (especially in the domain of predictors and analysis). In addition, as example of applicability in the real-world, a free-use prediction algorithm developed through AI failed to predict 1-year mortality post-HTx in cases from our center.

While AI-based prognostic and diagnostic functions performed better than those developed by traditional statistics, risk of bias, lack of external validation, and relatively poor applicability, may affect AI-based tools. More unbiased research with high quality BD meant for AI, transparency and external validations, are needed to have medical AI as a systematic aid to clinical decision making in HTx.

Racial disparities in outcomes after cardiac surgery are well reported. We sought to determine whether variation by race exists in controllable practices during coronary artery bypass graft surgery (CABG). We hypothesized that racial disparities exist in CABG quality metrics, but have improved over time.

All patients undergoing isolated CABG (2000 to 2019) in a multiple state database were stratified into three eras by race. Analysis included propensity matched White Americans and Black Americans. Primary outcomes included left internal mammary artery use, multiple arterial grafting, revascularization completeness, and guideline-directed medication prescription.

Of 72 248 patients undergoing CABG, Black American patients (n = 10 270, 15%) had higher rates of diabetes mellitus, hypertension, prior stroke, and myocardial infarction. After matching, 19 806 patients (n = 9903 per group) were well balanced. Left internal mammary artery use was significantly different early (era 1, Black Americans 84.7% vs White Americans 86.6%; P = .03), but equalized over time. Importantly, multiarterial grafting differed between Black Americans and White Americans over the entire study (9.1% vs 11.5%, P < .001) and within each era. Black Americans had more incomplete revascularization during the study period (14% vs 12.8%, P = .02) driven by a large disparity in era 1 (9.5% vs 7.2%, P < .001). Despite similar rates of preoperative use, Black Americans were more often discharged on a regimen of β-blockers (91.8% vs 89.6%, P < .001).

Coronary artery bypass graft surgery metrics of left internal mammary artery use and optimal medical therapy have improved over time and are similar despite patient race. Black Americans undergo less frequent multiarterial grafting and greater discharge β-blocker prescription. Identifying changes in controllable CABG quality practices across races supports a continued focus on standardizing such efforts.

Composite performance measures for the Society of Thoracic Surgeons (STS) Adult Cardiac Surgery Database participants (typically hospital departments or practice groups) are currently available only for individual procedures. To assess overall participant performance, STS has developed a composite metric encompassing the most common adult cardiac procedures.

Analyses included 1-year (July 1, 2018 to June 30, 2019) and 3-year (July 1, 2016 to June 30, 2019) time windows. Operations included isolated coronary artery bypass grafting (CABG), isolated aortic valve replacement (AVR), isolated mitral valve repair (MVr) or replacement (MVR), AVR + CABG, MVr or MVR + CABG, AVR + MVr or MVR, and AVR + (MVr or MVR) + CABG. The composite was estimated using Bayesian hierarchical models with risk-adjusted mortality and morbidity end points. Star ratings were based upon whether the 95% credible interval of a participant’s score was entirely lower than (1 star), overlapping (2 star), or higher than (3 star) the STS average composite score.

The North American procedural mix in the 3-year study cohort was as follows: 448 569 CABG, 72 067 AVR, 35 708 MVr, 29 953 MVR, 45 254 AVR + CABG, 12 247 MVr + CABG, 10 118 MVR + CABG, 3743 AVR + MVr, 6846 AVR + MVR, and 3765 AVR + (MVr or MVR) + CABG. Mortality and morbidity weightings were similar for 1- and 3-year analyses (76% and 24% [3-year]), as were composite score distributions (median, 94.7%; interquartile range, 93.6% to 95.6% [3-year]). The 3-year time frame was selected for operational use because of higher model reliability (0.81 [0.78-0.83]) and better outlier discrimination (26%, 3 star; 16%, 1 star). Risk-adjusted outcomes for 1-, 2-, and 3-star programs were 4.3%, 3.0%, and 1.8% mortality and 18.4%, 13.4%, and 9.7% morbidity, respectively.

The STS participant-level, multiprocedural composite measure provides comprehensive, highly reliable, overall quality assessment of adult cardiac surgery practices.

The Society of Thoracic Surgeons (STS) original coronary artery bypass graft surgery (CABG) composite measure uses a 1-year analytic cohort and 98% credible intervals (CrI) to classify better than expected (3-star) performance or worse than expected (1-star) performance. As CABG volumes per STS participant (eg, hospital or practice group) have decreased, it has become more challenging to classify performance categories using this approach, especially for lower volume programs, and alternative approaches have been explored.

Among 990 STS Adult Cardiac Surgery Database participants, performance classifications for the CABG composite were studied using various analytic cohorts: 1 year (current approach, 2017); 3 years (2015 to 2017); last 450 cases within 3 years; and most recent year (2017) plus additional cases to 450 total. We also compared 98% CrI with 95% CrI (used in other STS composite measures).

Using 3 years of data and 95% CrIs, 113 of 990 participants (11.4%) were classified 1-star and 198 (20%) 3-star. Compared with 1-year analytic cohorts and 98% CrI, the absolute and relative increases in the proportion of 3-star participants were 14 percentage points and 233% (n = 198 [20%] vs n = 59 [6%]). Corresponding changes for 1-star participants were 6.5 percentage points and 133% (n = 113 [11.4%] vs n = 48 [4.9%]). These changes were particularly notable among lower volume (fewer than 199 CABG per year) participants. Measure reliability with the 3-year, 95% CrI modification is 0.78.

Compared with current STS CABG composite methodology, a 3-year analytic cohort and 95% CrI increases the number and proportion of better or worse than expected outliers, especially among lower-volume Adult Cardiac Surgery Database participants. This revised methodology is also now consistent with other STS procedure composites.