Article Text

Download PDFPDF

Original research article
Performance of the BMJ learning training modules for ECG interpretation in athletes
  1. Nathan R Riding1,
  2. Jonathan A Drezner2
  1. 1 Athlete Health and Performance Research Centre, Aspetar Orthopaedic and Sports Medicine Hospital, Doha, Qatar
  2. 2 Department of Family Medicine, University of Washington, Seattle, Washington, USA
  1. Correspondence to Jonathan A Drezner, Department of Family Medicine, University of Washington, Seattle, WA 98195, USA; jdrezner{at}uw.edu

Abstract

To assess the accuracy of interpreting the athlete’s ECG both pre and post a series of online training modules among a range of healthcare professionals. 10 512 healthcare professionals from 138 different nations commenced the online course. These were primarily doctors (43%), nurses (18.4%) and other healthcare professionals (3.9%). The users came from 102 different specialities, with General Practice/Family Medicine (24.5%), Cardiology (10.6%), Emergency Medicine (8.7%) and Sports Medicine (6.6%) predominating. Among the 2023 users who completed both the pre-course and post-course test, there was an overall improvement of 15.3% (95% CI 13.9% to 16.6%; p<0.001). 930 completed all four other modules, and these users fared significantly better (18.7% increase; 95% CI 17.3 to 20.0) than those completing no additional modules (11.7% increase; 95% CI 3.3 to 17.7, p=0.036). Demographic analysis showed that while the starting pre-test scores varied significantly between profession/specialty groups (57.8%–82.6%), post-test scores were largely consistent (80.8%–84.6%). Although users showed the most improvement when interpreting primary electrical diseases (12.4% increase), it was also an area of notable weakness compared with the modules of normal training-related findings and cardiomyopathies. With the evolving criteria for ECG interpretation eliciting ever improving levels of specificity and sensitivity in the detection of conditions associated with sudden cardiac death among athletes, training is required to ensure the infrastructure and personnel is in place to uphold these standards. The BMJ Learning course presented is a valuable first step and demonstrates that such an online tool can be effective in aiding ECG interpretation among healthcare professionals globally.

  • ECG/electrocardiogram
  • Ehealth/telemedicine/mobile health
  • competency evaluation
  • educational goals and objectives

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

Cardiovascular preparticipation screening for athletes is widely advocated and within elite sport is implemented by over half of all International Sporting Federations prior to their flagship events.1 Cardiovascular screening is aimed at the identification of athletes with conditions predisposing to sudden cardiac death (SCD); however, the optimal screening protocol and use of a resting ECG remain areas of significant controversy.2–4 While general support for cardiovascular screening is based on medical, legal and ethical grounds, ECG antagonists nevertheless proclaim that the false-positive rate, expense and lack of trained personnel to interpret the findings are justification for the exclusion of the ECG within this process. Over the past decade, however, significant improvements in the false-positive rate of ECG investigation (2.2%–4.5%) has been achieved through application of contemporary ECG interpretation guidelines.5–8

The primary goal for ECG interpretation guidelines is to assist the physician in distinguishing physiological training-related ECG findings from training-unrelated ECG abnormalities that warrant further investigation. While the latest international consensus standards for ECG interpretation in athletes may further improve interpretation accuracy,9 10 low false-positive rates are demonstrated in studies using expert clinicians, and it is uncertain whether such refinements transfer more broadly to the practising physician responsible for their team or federation.

To date, limited data are available demonstrating the performance of ECG interpretation in the real-world setting.11–14 Initial studies demonstrated that the use of standardised ECG criteria assisted interpretation irrespective of specialty, with specificity improving 8%–11% and sensitivity 3%–5%.11 12 Recent work also suggests there is considerable variation in the interpretation of the athlete’s ECG, with inexperienced cardiologists 44% more likely to deem an ECG as abnormal than their experienced counterparts.15 Research also suggests that 45% of cardiologists tasked with interpreting athlete ECGs failed to use any specific criteria.16

The inconsistency of ECG interpretation among physicians underscores the need for continued education and training. In an effort to allow physicians and other healthcare specialists from around the world to gain a common foundation in ECG interpretation in athletes, members of the Seattle criteria17 writing group developed a series of freely accessible online training modules, provided by BMJ Learning and supported by the American Medical Society for Sports Medicine and FIFA.

The aim of the present study is to assess the performance of the BMJ Learning ECG interpretation training modules by comparing pre-online and post-online training course test results and identifying areas of strength and weakness.

Methods

Online course structure

The BMJ Learning ‘ECG interpretation in athletes’ course was composed of six modules and released online on 11 January 2013 (table 1). The modules were based on the 2013 Seattle criteria, with the aim of assisting physicians with ECG interpretation and the cardiovascular care of athletes, and provide lessons for healthcare professionals in interpreting the athlete’s ECG to distinguish normal physiological findings from pathological ECG patterns. Each module was accredited one Continuing Professional Development point from BMJ Learning and 17 other accrediting bodies worldwide.

Table 1

BMJ Learning Course—ECG interpretation in athletes: outline and descriptive data per module

To evaluate current knowledge and test any associated improvement, there was both pre-course (‘identify your learning needs’) and post-course (‘test your knowledge’) test modules. These comprised 10 questions that a physician may face in the applied setting. Representing the decisions a physician will have to make, the two possible choices were to classify the ECG presented as normal (not requiring further evaluation) or abnormal (further investigation indicated).

Between the pre-course and post-course test modules, three educational modules were created intending to explore the particulars of ECG interpretation in athletes. The first individually focused on the normal ECG findings associated with athletic training. The second and third content modules were specific to the diagnostic interpretation of cardiomyopathies and primary electrical diseases, respectively. These three modules each contained five true/false questions to be answered both prior to and post the educational material. A final module titled ‘utilising standardised criteria’ was added at a later date (4 July 2013) and comprised 15 standalone questions and difficult ECG examples intended to be completed using a summary sheet of the Seattle interpretation criteria.

On registration, users had to document their profession (ie, nurse, physician, etc.) and primary specialty (ie, family medicine, cardiology, etc.). Given the array of possible combinations, only the most common profession–specialty groupings were selected for statistical analysis. Each of the six modules could be completed individually and in any order.

Statistical analysis

Statistical analysis was conducted for users who completed both the pre-course and post-course test modules (‘identify your learning needs’ and ‘test your knowledge’) using SPSS (V.21.0, 2012; IBM). Data were expressed as mean, percentage change (Δ) and 95% CIs. Student’s t-test was used to determine whether there was a significant pre–post test score difference. An analysis of variance was used to establish the significance between profession–specialties, geographical region and the number of other modules completed, with Bonferroni adjustments. A p value <0.05 was considered statistically significant.

All individual identifying information was removed before the data were provided by BMJ Learning. Instead, each user was assigned a unique identification number to be tracked across the course modules. Institutional review board exemption was obtained by the Qatar Anti-Doping Laboratory ethics committee.

Results

Demographics of users

Between 11 January 2013 and 28 June 2016, there were 10 512 unique users registered onto the BMJ Learning ‘ECG interpretation in athletes’ online course (http://learning.bmj.com/learning/course-intro/.html?courseId=10042239). The most common professions reported were doctor (43%), nurse (18.4%) and other heart care professional (3.9%), with 22.3% of users leaving this field blank. A total of 102 different primary specialties were registered. The most common specialty was Family Medicine/General Practice with 2579 users (24.5%). The specialties of Cardiology (1115, 10.6%), Emergency Medicine (912, 8.7%), Sports Medicine (691, 6.6%) and General Internal Medicine (547, 5.2%) were the other most common specialties.

There was a broad geographical representation with individuals from 138 different nations completing at least one module (figure 1). Those from North Europe predominated with 3826 (36.4%) users, followed by West Asia (2952, 28%), North America (1507, 14.3%), South Asia (496, 4.7%), South Europe (384, 3.7%) and Australasia (309, 2.9%).

Figure 1

Global representation of users undertaking the online course. Shaded areas represent countries with course participants.

Overall course statistics

Of the 10 512 unique users, 5059 completed the course pre-test ‘Identify your learning needs’. The subsequent three mid modules involving course content were completed with varying rates; 3265 completing ‘Normal ECG findings’, 4545 completing ‘Abnormal ECG findings in athletes: recognising changes suggestive of cardiomyopathy’ and 2741 completing ‘Abnormal ECG findings in athletes: recognising changes suggestive of primary electrical disease’. Less than half of those who took the first module completed the post course ‘Test your knowledge’ unit (2023, 40%). Module 5 ‘Utilising standardised criteria’ in which users were encouraged to use a summary sheet of the Seattle criteria had 2045 completions. Only 930 (8.9%) participants completed all six modules, with the majority (5929, 56.4%) completing only one module.

Overall improvement in ECG interpretation

From the 1549 users who completed both ‘Identify your learning needs’ and ‘Test your knowledge’ modules, there was an overall improvement from 67.2% to 82.4% (Δ 15.3%, 95 % CI 13.9 to 16.6, p<0.001). The greatest improvement was found among individuals completing all four other modules (Δ 18.7%, 95% CI 17.3 to 20.0), compared with those completing no (Δ 10.5%, 95% CI 3.3 to 17.7, p=0.030), one (Δ 11.7%, 95% CI 6.6 to 16.9, p=0.053), two (Δ 14.4%, 95% CI 8.8 to 20.0, p=0.454) or three (Δ 7.6%, 95% CI 3.6 to 11.5, p<0.001) other modules.

Demographic impact on course performance

The greatest pre to post improvement came from users from Asia (Δ 22.5%, 95% CI 19.7 to 25.3) (figure 2). This was predominantly attributed to the 25.2% increase from the 313 Western Asia users. The improvement among Asian users was significantly greater than that seen among all other regions (p<0.05) except South America. While no statistical significance was found between any other regions, South American users improved by 18.8% (95% CI 10.9 to 26.6), North American users by 15.7% (95% CI 13.8 to 17.6), Europeans by 11.1% (95% CI 8.9 to 13.4), Oceania users by 11.0% (95% CI 2.3 to 19.7) and Africans by 5.8% (95% CI −2.7 to 14.3). The continent with the greatest post-test score was North America with an average score in the ‘Test your knowledge’ module of 85.3%. The lowest post-test mean was in African users (77.1%).

Figure 2

Percentage improvement from the modules ‘Identify your learning needs’ to ‘Test your knowledge’ categorised by user’s home continent.

Regarding profession and specialty comparisons, the best performing group were nurses specialising in cardiology, with a final post-test average of 88.4% (95% CI 84.7 to 92.1), followed by acute internal medicine doctors (84.6%, 95% CI 81.5 to 87.8) and family medicine/general practice and sports medicine doctors (both 84.4%, 95% CI 82.0 to 86.7) (figure 3). The greatest pre-test to post-test improvement was in family medicine nurses (60.8% to 83.8%, Δ 23%); however, this change was only statistically significantly greater compared with cardiologists (p<0.001) whose pre-test to post-test improvement was only 0.4% (82.6% to 83.0%). However, among cardiologists, those who completed all six modules had the highest post-course scores (88.7%, 95% CI 85.3 to 92.1) and greatest overall improvement from pre-test (Δ 4.6%).

Figure 3

Percentage improvement from the modules ‘Identify your learning needs’ to ‘Test your knowledge’ categorised by profession–specialty.

General strengths and weaknesses within the ECG interpretation modules

Within the pre to post modules, there was a tendency for users to perform better at identifying abnormalities associated with structural cardiac disease as opposed to conduction disturbances. The question most commonly answered correctly (92.5%) examined the identification of findings suggestive of cardiomyopathy (figure 4). Although used as a pre-test question, only 31.3% of users identified that marked sinus arrhythmia was a normal finding in athletes (figure 5). Recognising Mobitz type 1 (Wenckebach) (55%), junctional escape rhythm (65%) and premature atrial contractions (65%) as normal findings were other low-performing questions in later modules.

Figure 4

Question from ‘Identify your learning needs’, which users most commonly answered correctly (92.5%). Presented is the ECG of a young athlete with a family history of heart disease. In your opinion, does this patient require further testing for cardiomyopathy? Yes or No. (The ECG demonstrates left bundle branch block, which is an abnormal finding and requires a comprehensive evaluation to exclude myocardial disease. This athlete was diagnosed with hypertrophic cardiomyopathy.)

Figure 5

Question from ‘Identify your learning needs’, which users most commonly answered incorrectly (31.3% correct). The ECG is taken from a long-distance runner in a yearly check-up. He is asymptomatic without prior episodes of palpitations, collapse, chest pain or exercise intolerance and has nothing of note from his family history. Which of the following statements is correct regarding this ECG? (1) Normal; (2) ECG shows atrial fibrillation that requires further investigation; (3) ECG shows Mobitz type I second-degree heart block that requires no further investigation; (4) ECG is an abnormal ECG for an athlete and requires further investigation. (The ECG demonstrates marked sinus arrhythmia, which is a normal finding in athletes. Note that sinus arrhythmia is not listed as an answer choice, which may have contributed to the low percentage of correct responses.)

This was echoed within the three mid modules, whereby the pre module mean was lowest within the primary electrical diseases section (69.3%). After undertaking the module, the test score improved to 81.7%, representing the greatest improvement (12.4%) of any of the three content modules. Within this section was one of the best performing questions (89.2% correct), which was utilising the newly amended criteria and identifying that a QTc beyond 470 ms in men and 480 ms in women was considered the cut-off for a long QT interval warranting further testing.

Users also showed improvement within the other two content modules, increasing the normal ECG module mean score by 11.9% (76.2% to 88.1%) and the cardiomyopathy mean score by 8.9% (75.4% to 84.3%). The module ‘Utilising standardised criteria’ intentionally provided more challenging ECG examples and instructed users to complete it with the aid of the guidelines. While there was a slight tendency for users to correctly identify abnormalities suggesting structural disease (74.4%) over primary electrical diseases (72.3%), the difference was small with an overall average score of 73.8%.

Discussion

As we attempt to uncover the inherent benefits and risks of sport and exercise participation, there has been a notable emergence of sports cardiology as an independent specialty. The continued global adoption of cardiovascular screening requires that measures must be in place to ensure the accurate and effective implementation of ECG if used. This should fundamentally start with training physicians to understand and identify the nuances of an athlete’s ECG.

Can E-learning aid athlete ECG understanding?

E-learning is an effective tool for enhancing ECG interpretation in general cardiology18–20 and in the effort to facilitate understanding of the athlete’s heart. Thus, the development of an online educational platform for ECG interpretation in athletes represents an important step.

The BMJ Learning course demonstrates that such a programme can considerably aid the healthcare professional’s interpretation of the athlete’s ECG, irrespective of their profession, specialty and nationality. The 15.3% overall improvement in ECG interpretation from pre-test to post-test documents the importance of educating the clinicians responsible for the cardiac health of athletes. Studies from Drezner et al 11 and Exeter et al 12 also highlight significant improvement in interpretation following the provision of an ECG criteria tool with no additional training.

It is possible the high proportion of abnormal ECGs within these studies (up to 30%) is not applicable to a real-world scenario where a ‘prevalence effect’ makes the identification of a rare abnormal finding, typically around 5%, more challenging when interspersed among the vast number of normal ECGs.21 Indeed, Brosnan et al 13 found only modest agreement among experienced physicians when interpreting 440 consecutively collected ECGs. Therefore, despite notable improvements found in the present study, complementary education, training and practical experience are still a necessary adjunct to achieve a higher level of proficiency.

Challenges to ECG interpretation

Though there was improvement from pre to post tests, there was considerable variability in performance depending on the specific manifestation represented on the ECG. Throughout the modules, there was a trend for users to perform worse in identifying ECG abnormalities relating to primary electrical diseases, echoing previous findings and with important relevance to clinical practice.22 From a clinical perspective, sudden arrhythmic death syndrome that may sometimes represent undiagnosed electrical disorders has been attributed to 42% of SCD in athletes from the UK.23

Correctly recognising normal physiological variants in the resting rhythm was challenging for clinicians in this study. While computerised ECG interpretation is evolving, its ability to correctly identify non-sinus rhythm is frequently incorrect, being as low as 53.5%.24 Thus, this study highlights that additional education and training on the physiological adaptations in athletes is needed to reduce false-positive interpretations.

Do we have the infrastructure in place?

Within the general cardiology community, there is an emerging lack of suitably qualified physicians capable of interpreting ECGs.25 Similar concerns are present within sports medicine and sports cardiology regarding the interpretation of an athlete’s ECG. Current research has demonstrated, however, that correct identification of abnormal ECGs within athletes is high and with low false-positive rates when interpreted at expert centres with substantial sports cardiology experience.5 7 8 This may distort the true state of our current infrastructure as it is often the role of the team physician or general cardiology consultant to interpret the findings.

Berte et al 14 suggested there is a discrepancy between ECG interpretation accuracy between sports medicine physicians and cardiologists. Among cardiologists, Dhutia et al 15 found that both ECG interpretation accuracy and the choice of further investigations were heavily physician dependent, with those experienced in athlete screening performing better than their inexperienced colleagues. The BMJ Learning course was aimed at the novice interpreter, but little inter-profession/specialty difference in accuracy was found.

Current consensus guidelines recommend that the responsible physician for the cardiovascular care of athletes work in close collaboration with local cardiology services.3 It is now common place within Major League Soccer and the English Premier League to have a team cardiologist, with other professional sporting organisations forming partnerships between league medical staff and local cardiologists.26 Nevertheless, our data support that further education is needed to produce a larger substructure of physicians skilled and capable of accurate ECG interpretation in athletes.11

Both the European Society of Cardiology27 and the American College of Cardiology28 have dedicated sports cardiology sections within their remit and as of this year have each outlined core curricula to detail the necessary skills and knowledge required to practise effective sports cardiology. Both cardiology bodies endorse additional training of cardiologists for this role given reference centres are not established in many areas for such undertaking. In addition, it is often the sports medicine physician who will undertake the cardiovascular screening of athletes. Although Asif and Drezner29 outline recommendations for sports cardiology training for the sports medicine physician, including performing at least 100 preparticipation cardiovascular screenings and the interpretation of at least 500 athlete ECGs, there is currently no universal standard for sports cardiology training within sports medicine.

Global variation with ECG interpretation

With over 10 000 users from 138 different nations registering for the online course, global interest in the field is evident. Aside from Northern Europe, the most engaging region was West Asia, which also demonstrated the greatest pre-course to post-course increase in accuracy (25.2%). The two continents with the fewest registered users were South American and Africa. SCD is a global problem and the low participation from these two continents highlights the importance to recognise SCD is not merely a European/North American issue and more must be done to engage physicians from other regions, especially considering athletes of African descent have the highest burden of SCD worldwide.30

Limitations

This study has several limitations. First, it is important to highlight that the proportion correct on any of the modules does not represent the accuracy of ECG screening in clinical practice. In other words, the collection of ECGs presented in the modules should not be construed as the relative proportion of normal or abnormal ECGs in a young athletic population undergoing cardiovascular screening. Thus, the true impact of online education and potential value of applying standardised interpretation criteria by clinicians with various experience and backgrounds is not determined by this study. This study reports descriptive statistics from an online training course taken by a large group of self-selected multinational and multidisciplinary healthcare professionals and was not designed to be a randomised controlled trial. Retention of information was also not assessed, an important caveat given some evidence suggests ECG interpretation test scores reduce 2–4 weeks post-education.20 In addition, other factors that may have influenced individual performance, such as prior experience in ECG interpretation in athletes, other educational courses completed or how long participants took to complete a module, are unknown. Lastly, the modules were designed for primary care physicians with an interest in sports medicine and thus targeted clinicians with more novice ECG interpretation skills. Indeed, cardiologists demonstrated the least amount of pre-test/post-test improvement, suggesting that more advanced training modules may be needed for physicians with greater ECG experience.

Conclusion

The prevention of SCD in athletes remains a high priority among both cardiology and sports medicine communities, and the education of physicians capable of accurate ECG interpretation is fundamental to this objective. While this study demonstrates that online education can be effective in aiding the interpretation of the athlete’s ECG among healthcare professionals worldwide, the online course is only a first step in the training process. Additional efforts towards continued education and increased clinical practice training and experience are warranted.

Key messages

What is already known on this subject?

  • There is little evidence of the real-world application of ECG interpretation in athletes and how structured education can impact accuracy.

  • Among small-scale studies, ECG interpretation accuracy improves with the provision of standardised criteria guidelines.

  • A lack of qualified personnel to accurately interpret the athlete’s ECG is a limitation of ECG-led preparticipation screening.

What might this study add?

  • Online education can be effective in aiding the interpretation of the athlete’s ECG among healthcare professionals worldwide.

  • A larger emphasis in distinguishing primary electrical diseases from normal electrical rhythms should be considered as newer versions of ECG training modules are developed.

How might this impact on clinical practic?

  • Online education can serve as an important starting point to gain fundamental knowledge for the interpretation of an athlete’s ECG.

  • Additional efforts towards education and increased clinical practice training and experience are warranted to advance the infrastructure of clinicians capable of accurate ECG interpretation in athletes.

References

Footnotes

  • Contributors NRR and JAD made substantial contributions to the conception or design of the work and/or the acquisition, analysis or interpretation of data for the work; drafting the work, revising it critically for important intellectual content and final approval of the version to be published; and agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient consent Not required.

  • Provenance and peer review Not commissioned; externally peer reviewed.