Article Text

Download PDFPDF
Original research
Automatic detection of cardiac conditions from photos of electrocardiogram captured by smartphones
  1. Chun-Ka Wong,
  2. Yuk Ming Lau,
  3. Hin Wai Lui,
  4. Wai Fung Chan,
  5. Wing Chun San,
  6. Mi Zhou,
  7. Yangyang Cheng,
  8. Duo Huang,
  9. Wing Hon Lai,
  10. Yee Man Lau,
  11. Chung Wah Siu
  1. Cardiology Division, Department of Medicine, School of Clinical Medicine, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong SAR, China
  1. Correspondence to Dr Chun-Ka Wong, Division of Cardiology, Department of Medicine, The University of Hong Kong, Hong Kong, Hong Kong; emmanuelckwong{at}gmail.com; Dr Chung Wah Siu; cwdsiu{at}hku.hk

Abstract

Background Researchers have developed machine learning-based ECG diagnostic algorithms that match or even surpass cardiologist level of performance. However, most of them cannot be used in real-world, as older generation ECG machines do not permit installation of new algorithms.

Objective To develop a smartphone application that automatically extract ECG waveforms from photos and to convert them to voltage-time series for downstream analysis by a variety of diagnostic algorithms built by researchers.

Methods A novel approach of using objective detection and image segmentation models to automatically extract ECG waveforms from photos taken by clinicians was devised. Modular machine learning models were developed to sequentially perform waveform identification, gridline removal, and scale calibration. The extracted data were then analysed using a machine learning-based cardiac rhythm classifier.

Results Waveforms from 40 516 scanned and 444 photographed ECGs were automatically extracted. 12 828 of 13 258 (96.8%) scanned and 5399 of 5743 (94.0%) photographed waveforms were correctly cropped and labelled. 11 604 of 12 735 (91.1%) scanned and 5062 of 5752 (88.0%) photographed waveforms achieved successful voltage-time signal extraction after automatic gridline and background noise removal. In a proof-of-concept demonstration, an atrial fibrillation diagnostic algorithm achieved 91.3% sensitivity, 94.2% specificity, 95.6% positive predictive value, 88.6% negative predictive value and 93.4% F1 score, using photos of ECGs as input.

Conclusion Object detection and image segmentation models allow automatic extraction of ECG signals from photos for downstream diagnostics. This novel pipeline circumvents the need for costly ECG hardware upgrades, thereby paving the way for large-scale implementation of machine learning-based diagnostic algorithms.

  • electrocardiography

Data availability statement

Data are available on reasonable request. Anonymised data will be available from the corresponding author on reasonable request up to 3 years after publication of the article. Raw images of the ECGs could not be transferred to third parties owing to patient privacy concerns.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Data availability statement

Data are available on reasonable request. Anonymised data will be available from the corresponding author on reasonable request up to 3 years after publication of the article. Raw images of the ECGs could not be transferred to third parties owing to patient privacy concerns.

View Full Text

Footnotes

  • C-KW, YuML and HWL are joint first authors.

  • X @ChunKaWong

  • Contributors C-KW, Yuk-Ming L, HWL and CWS conceived and designed the study. C-KW, Yuk-Ming L, HWL, WFC, WCS and CWS designed and created the software and deep learning systems. WHL, Yee-Man L, DH, MZ and YC contributed to data acquisition. C-KW and CWS reviewed and labelled cardiac rhythm of ECG. C-KW, Yuk-Ming L, HWL, WFC, WCS and CWS performed data analysis and interpretation. C-KW, Yuk-Ming L, HWL, WFC, WCS and CWS wrote the first draft of the manuscript. CWS revised the manuscript critically for important intellectual content. All authors have read and approved the final version of the manuscript to be published. C-KW is author responsible for overall content as the guarantor.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.