Article Text

Download PDFPDF
Original research
Incremental value of machine learning for risk prediction in tetralogy of Fallot
  1. Ayako Ishikita1,
  2. Chris McIntosh1,2,3,
  3. S Lucy Roche1,
  4. David J Barron4,
  5. Erwin Oechslin1,
  6. Lee Benson5,
  7. Krishnakumar Nair1,
  8. Myunghyun M Lee4,
  9. Michael N Gritti5,
  10. Kate Hanneman3,
  11. Gauri Rani Karur3,
  12. Rachel M Wald1,3,5
  1. 1 Division of Cardiology, Peter Munk Cardiac Centre, University Health Network, University of Toronto, Toronto, ON, Canada
  2. 2 Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
  3. 3 Joint Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada
  4. 4 Department of Cardiovascular Surgery, Hospital for Sick Children, University of Toronto, Toronto, ON, Canada
  5. 5 Division of Cardiology, Hospital for Sick Children, University of Toronto, Toronto, ON, Canada
  1. Correspondence to Dr Rachel M Wald, Peter Munk Cardiac Centre, Cardiology, University of Toronto, Toronto, ON M5G 2N2, Canada; Rachel.Wald{at}uhn.ca

Abstract

Objective Machine learning (ML) can facilitate prediction of major adverse cardiovascular events (MACEs) in repaired tetralogy of Fallot (rTOF). We sought to determine the incremental value of ML above expert clinical judgement for risk prediction in rTOF.

Methods Adult congenital heart disease (ACHD) clinicians (≥10 years of experience) participated (one cardiac surgeon and four cardiologists (two paediatric and two adult cardiology trained) with expertise in heart failure (HF), electrophysiology, imaging and intervention). Clinicians identified 10 high-yield variables for 5-year MACE prediction (defined as a composite of mortality, resuscitated sudden death, sustained ventricular tachycardia and HF). Risk for MACE (low, moderate or high) was assigned by clinicians blinded to outcome for adults with rTOF identified from an institutional database (n=25 patient reviews conducted by five independent observers). A validated ML model identified 10 variables for risk prediction in the same population.

Results Prediction by ML was similar to the aggregate score of all experts (area under the curve (AUC) 0.85 (95% CI 0.58 to 0.96) vs 0.92 (0.72 to 0.98), p=0.315). Experts with ≥20 years of experience had superior discriminative capacity compared with <20 years (AUC 0.98 (95% CI 0.86 to 0.99) vs 0.80 (0.56 to 0.93), p=0.027). In those with <20 years of experience, ML provided incremental value such that the combined (clinical+ML) AUC approached ≥20 years (AUC 0.85 (95% CI 0.61 to 0.95), p=0.055).

Conclusions Robust prediction of 5-year MACE in rTOF was achieved using either ML or a multidisciplinary team of ACHD experts. Risk prediction of some clinicians was enhanced by incorporation of ML suggesting that there may be incremental value for ML in select circumstances.

  • Tetralogy of Fallot
  • Magnetic Resonance Imaging
  • Quality of Health Care
  • Risk Factors

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Data availability statement

Data are available upon reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request.

View Full Text

Footnotes

  • Twitter @EOechslin, @michael.gritti, @drrachelwald

  • Contributors RMW has responsible for the overall content as the guarantor. AI, CM and RMW substantially contributed to the study conceptualisation. AI, CM, SLR, DJB, EO, LB, KN, MML, MNG, KH, GRK and RMW contributed to data analysis and interpretation. AI drafted the original manuscript. RMW supervised the conduct of this study. All authors critically reviewed and revised the manuscript draft and approved the final version for submission.

  • Funding This research was funded by the Canadian Institutes of Health Research (MOP-119353) to RW.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.