Objective To evaluate the performance of a new electronic screening tool (TARB-Ex) in detecting general practice patients at potential risk of familial hypercholesterolaemia (FH).
Methods Medical records for all active patients seen between 2012 and 2014 (n=3708) at a large general practice in Perth, Western Australia were retrospectively screened for potential FH risk using TARB-Ex. Electronic extracts of medical records for patients identified with potential FH risk (defined as Dutch Lipid Clinic Network Criteria (DLCNC) score ≥5) through TARB-Ex were reviewed by a general practitioner (GP) and lipid specialist. High-risk patients were recalled for clinical assessment to determine phenotypic FH diagnosis. Performance was evaluated against a manual record review by a GP in the subset of 360 patients with high blood cholesterol (cholesterol ≥7 mmol/L or low-density lipoprotein cholesterol ≥4.0 mmol/L).
Results Thirty-two patients with DLCNC score ≥5 were identified through electronic screening compared with 22 through GP manual review. Sensitivity was 95.5% (95% CI 77.2% to 99.9%), specificity was 96.7% (95% CI 94.3% to 98.3%), negative predictive accuracy was 99.7% (95% CI 98.3% to 100%) and positive predictive accuracy was 65.6% (95% CI 46.9% to 8%). Electronic screening was completed in 10 min compared with 60 h for GP manual review. 10 of 32 patients (31%) were considered high risk and recalled for clinical assessment. Six of seven patients (86%) who attended clinical assessment were diagnosed with phenotypic FH on examination.
Conclusions TARB-Ex screening is a time-effective and cost-effective method of systematically identifying potential FH risk patients from general practice records for clinical follow-up.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Familial hypercholesterolaemia (FH) is an autosomal dominant inherited disorder characterised by excessively high plasma levels of low-density lipoprotein cholesterol (LDL-c) from birth due to a defect in the LDL receptor pathway.1 Without early detection and treatment, individuals with FH are at significantly elevated risk of premature coronary artery disease due to accelerated onset of atherosclerosis.2 FH is relatively easy to detect and treat but only an estimated 10% of FH cases are currently detected.3 Poor detection rates in Australia are largely due to a lack of any systematic population-based method of FH screening at present.4
Genetic testing has previously been the diagnostic gold standard for FH. However, phenotypic diagnosis is now recommended for use in the Australian primary care setting as a more pragmatic approach.5 ,6 Within primary care, general practice has been increasingly identified as an optimal setting for FH screening5–12 with suggested models including universal screening of children,13 opportunistic screening of patients with a positive family history and/or elevated cholesterol14 and screening using electronic medical records (EMRs).11 ,12 ,15 Of these, electronic screening may offer the most systematic and cost-effective approach with greatest potential for integration into existing clinical practice.4 ,15 More than 90% of Australian general practitioners (GPs) now maintain EMRs to reduce medical errors and enhance quality-of-care.16 This in turn has provided a rich pool of data that enables development of electronic tools to increase clinical efficiency.
Three studies of electronic screening methods for FH risk have been published. In the first, Gray et al15 used electronic searches to identify patients with FH indicators at a large primary care centre with manual review and scoring for FH risk according to the Dutch Lipid Clinic Network Criteria (DLCNC17). Kirke et al4 used community pathology laboratory and general practice patient databases to screen for patients with FH indicators and follow-up clinical assessment with a FH nurse. Most recently, Weng et al18 developed the familial hypercholesterolaemia case ascertainment tool (FAMCAT) algorithm to predict the probability of FH in general practice patients using indicators routinely available in EMRs. All studies showed electronic screening was able to screen large numbers of patients and identify new index cases of FH. However, no method is currently optimised for integration into normal clinical practice.
Our study aimed to develop and evaluate the performance of an electronic screening tool that uses routine general practice database information to identify patients with high DLCNC scores for further clinical investigation. We internally validated our electronic screening tool against a manual review of patient records for FH risk by an independent GP.
Setting and design
The study was a retrospective investigation of FH risk in active patients seen between 2012 and 2014 at a large metropolitan general practice in Perth, Western Australia.
De-identified information was analysed but the treating practice was able to reidentify patients for clinical follow-up if required.
We developed an electronic screening tool (TARB-Ex) that screens for FH risk in general practice. TARB-Ex extracts routine clinical information from EMRs to derive a DLCNC score and identify patients with potential FH risk for clinical investigation. TARB-Ex was developed using Structured Query Language (SQL) technology and written for Best Practice19 clinical software.
Figure 1 displays the TARB-Ex screening process. First, active patients (≥3 visits within the last 2 years20) at the practice were identified. A subset of patients who ever had a pathology record of total cholesterol ≥7 mmol/L or LDL-c ≥4.0 mmol/L (DLCNC minimum for FH risk) was then extracted. Clinical and family histories for these patients were also extracted from both drop-down lists and free-text entries. Allowance was made for variation in spelling and typographical errors by creating a dictionary of possible terms. Prescription information was also extracted. Patients receiving a statin prescription >1 week and <6 months of highest LDL-c measurement were regarded as on statin therapy and a correction algorithm was applied to estimate the patient's cholesterol prior to treatment.21 Corrected LDL-c levels were used for further computation.
For each patient, TARB-Ex produces an extract containing each element of the DLCNC along with a computed DLCNC22 score. Patients with DLCNC score ≥5 were considered potential risk patients. While a DLCNC score of 5 indicates ‘possible’ FH, we adopted this more inclusive strategy given that general practice patient history records are generally poorly documented.16 Thus, a patient with a score of five with a missing family history record may become a ‘probable’ case (DLCNC scores 6–8) with a positive family history upon further assessment.
De-identified TARB-Ex extracts for patients with DLCNC scores ≥5 were reviewed by a lipid specialist and GP to exclude any secondary causes of hypercholesterolaemia (eg, liver or renal disease, diabetes mellitus, corticosteroid use or hypothyroidism). Patients considered at high risk of FH were recalled for clinical assessment with the GP and lipid specialist team. A detailed assessment of personal and family history and physical examination for arcus cornealis and tendinous xanthomata was undertaken. A family pedigree tree was also constructed highlighting cardiovascular events in first-degree relatives. Based on this, a DLCNC score and FH diagnosis was determined.
Validation of TARB-Ex
We evaluated TARB-Ex assessment for potential FH risk against manual review of patient EMRs. Manual review was undertaken by an independent, practice-based GP who received training in using the DLCNC. Study IDs for the subgroup of patients with elevated cholesterol were provided to the GP who then manually reviewed and derived a DLCNC based on the patient EMR. The GP was asked to remain objective in recording information and was blinded to the results of the TARB-Ex screening. Manual review was considered the gold standard method and determined true positives for potential FH risk.
Data were analysed using Stata V.13.1. All primary analyses were tested against an α level of 0.05 (uncorrected, two-tailed). Descriptive statistics are presented as mean±SD for continuous outcomes and frequencies for categorical variables. Independent samples Student's t tests and χ2 analyses were used to compare differences in continuous and categorical outcomes, respectively.
We compared agreement between the two methods on: (1) highest uncorrected LDL-c recorded, (2) DLCNC score and (3) potential FH risk category. For continuous outcomes, we assessed inter-rater agreement using the intraclass correlation coefficient (ICC; two-way mixed model, absolute agreement) with 95% CIs and Bland–Altman plots (plotting the difference between methods against GP manual review as the reference method). For comparison of agreement on FH risk category, we calculated sensitivity, specificity and positive and negative predictive values with 95% CIs for TARB-Ex against GP manual record review.
Total and per case screening costs between TARB-Ex and GP manual review were calculated and compared. Costs in $A were calculated for TARB-Ex development, data extraction by various staff (GP, practice manager and practice nurse), lipid specialist and GP review of TARB-Ex patient extracts and GP manual review. Costs were calculated as follows and inflated with consumer price index increases to 2015:23 (1) TARB-Ex development ($40/h postdoctoral researcher salary), (2) lipid specialist ($200/h24), (3) GPs $148/h based on the modelled activity profile for a four-doctor practice25 and annual workload of 46 weeks×24 h/week26–28 and (4) practice manager $34/h and practice nurse $30/h based on an annual workload of 46 weeks×37 h/week.25
A total of 3708 active patients attended the practice over the study period (53.3% female, mean age 43.5±24.6 years). Figure 2 displays the patient flow for electronic screening. Of all active patients, 1126 (30.4%) had at least one recorded LDL-c measurement. Patients with a recorded blood result were significantly older (65.6±14.1 years) than those never tested (33.8±21.9 years), p<0.001.
Three hundred and sixty (31.9%) active patients with blood records had elevated cholesterol. Thirty-nine patients were on statin treatment at the time of highest LDL-c measurement and required cholesterol correction. Statin strength data were not extractable in the EMR and so the lowest adjustment factor for each medication was applied as a conservative strategy. For example, the lowest correction factor for atorvastatin (ie, 10 mg) was applied for all patients with a script for ‘Lipitor Tablet’. Thirty-five patients had a recorded history of premature cardiovascular disease and two had premature ischaemic heart disease. Family history was missing for 65.2% (235/360) of patients. Of those with family history recorded, 27/125 (21.6%) had a positive family history of heart disease. Tendinous xanthomata and arcus cornealis were not recorded for any patients.
Thirty-two patients had DLCNC scores ≥5 and were identified as at potential risk of FH. One patient had an existing diagnosis of FH. The prevalence of patients at potential risk of FH was 1:116 of total active patients and 1:35 for the subgroup of active patients with a cholesterol measurement. Electronic screening took approximately 10 min to run.
GP manual record review identified 22 potential FH risk patients (DLCNC scores ≥5), a prevalence of 1:168 of total active patients and 1:51 of active patients with a cholesterol measurement. Manual review for the 360 patients was completed in approximately 60 h.
Comparison of TARB-Ex and manual review
Highest uncorrected LDL-c measures
Agreement on highest uncorrected LDL-c measures was high, ICC=0.95 (95% CI 0.92 to 0.96). The mean difference in LDL-c was 0.09 mmol/L, p=0.067 (figure 3A). All differences in LDL-c measures between the two methods were positive, indicating that TARB-Ex systematically identified higher recorded LDL-c measures for a given patient compared with manual review.
Agreement on DLCNC scores was also high, ICC=0.87 (95% CI 0.84 to 0.90). The mean difference in DLCNC scores between the two methods was 0.22, p=0.069 (figure 3B). Two distinct patterns were observed where score differences exceeded the limits of agreement: (1) TARB-Ex derived higher DLCNC scores where the manual review indicated very low FH risk (DLCNC ≤3) and (2) the manual review produced more DLCNC scores ≥8 compared with the electronic screening method.
Potential FH risk
Agreement between the two methods was 96.7% (348/360 patients) (table 1). Sensitivity was 95.5% (95% CI 77.2% to 99.9%) and specificity was 96.7% (95% CI 94.3% to 98.3%). The positive predictive value (PPV) was 65.6% (95% CI 46.9% to 81.4%) and negative predictive value was 99.7% (95% CI 98.3% to 100%).
Clinical follow-up and genetic testing
Following review of TARB-Ex extracts of medical records, 10 of 32 patients (31%) with DLCNC score ≥5 were considered at high risk of FH. Twenty-two patients were excluded at this stage due to overcorrection of LDL-c levels. That is, TARB-Ex had identified a statin prescription within the preceding 6 months of the patient's LDL-c measurement and applied a correction accordingly. However, upon review of the patient's complete prescription and blood history, it was considered that these patients were likely non-compliant with prescribed medication at the time of measurement and cholesterol adjustment was thus unnecessary. DLCNC scores for these 22 patients were revised down using the uncorrected LDL-c measurement. Review of records by the GP and lipid specialist for the 32 patients took approximately 1 h.
Table 2 compares DLCNC scores for the 10 patients recalled for clinical follow-up as assessed by TARB-Ex, GP manual review and at clinical assessment. Three patients were unable to attend assessment. Six of seven patients (86% sensitivity) who attended were diagnosed with phenotypic FH on clinical examination with one patient referred for genetic testing for FH. Four of 10 patients recalled for clinical follow-up were considered low FH risk according to the GP manual review. DLCNC scores were upgraded at clinical follow-up.
Costs of screening for the two methods are presented in table 3. Two cost estimates are presented for TARB-Ex screening: actual study costs and approximate running costs in general practice.
Our study trialled an innovative electronic screening tool for FH risk using general practice EMRs. TARB-Ex was developed for Best Practice19 clinical management software. However, it has the capacity to be adapted for any SQL-based practice software such as MedicalDirector, ZedMed, MedTech, Practix and Monet which would encompass approximately 90%–95% of software usage in Western Australia (personal communication). Overall, results suggest that electronic screening is a quick and accurate method for systematically identifying patients at potential risk of FH for further clinical investigation, while minimising GP workload and practice costs. Our electronic screening method showed high sensitivity, specificity and negative predictive power, demonstrating comparable performance with a GP manual review of patient records for FH risk in a much shorter time (10 min vs 60 h for manual review).
Thirty-two patients were assigned DLCNC scores ≥5 by TARB-Ex compared with 22 from the GP manual review accounting for the comparatively lower PPV at 65.6%. Traditional interpretation of the PPV would suggest that one-third of potential FH risk patients identified through TARB-Ex were falsely classed as such. However, this could reflect the increased accuracy of electronic screening. Distributions in Bland–Altman plots show that TARB-Ex systematically identified higher LDL-c measures compared with manual review. As this process required the GP to identify and record the highest ever uncorrected LDL-c test value for each of 360 patients with no clinical judgement, we use this finding as support for the increased accuracy of electronic screening. Moreover, four of 10 patients identified by TARB-Ex and recalled for clinical follow-up were assessed as having low FH risk according to manual review. Medical records can be extensive for a given patient and manual scanning can be prone to human error, especially if the size of patient records to be reviewed is large.
Overall, the study shows that electronic screening is an efficient and accurate method of assessing FH risk with DLCNC scores strongly correlating with clinical examination. Both manually reviewed and TARB-Ex assessed DLCNC scores were upgraded at clinical follow-up. In each instance, this was because family and/or clinical history and physical signs were not recorded in patient notes. The ultimate sensitivity achieved was 86% with six of seven patients examined diagnosed with phenotypic FH. One patient recalled for clinical assessment had their DLCNC score revised downwards after examination (patient 5). This patient had been tested for FH in the past and assessed to be an unlikely case on combination of genetic and clinical findings. Two patients who did not return for clinical assessment were determined to have the lowest FH risk by manual review. If these patients did not truly have FH, this could affect the ultimate sensitivity of TARB-Ex screening.
Compared with previous electronic screening methods for FH risk,4 ,15 this method is optimised for easy integration into normal clinical practice. Previous electronic screening models have been very time-consuming and resource-consuming. Each manual search of medical records in the Gray et al15 study took approximately 30 min, creating a total of 201 additional hours of work for the GP for the 402 patients requiring review. Kirke et al's4 methods involved many different staff and costs (eg, pathology technicians, GPs, nurses, data extraction and mail-out) with low response rates. Moreover, 93% of their patients who attended clinical assessment were subsequently assessed to have low FH risk (DLCNC score ≤5). The current method offers a more time-effective (<1 h) and cost-effective ($5–10 per potential diagnosis) model than previous models and when compared with manual medical review. Ease of use of the tool means a GP, practice nurse or practice manager can perform the screening prior to clinical recall. GP involvement is minimised to reviewing medical records for patients identified as at-risk, to excluding secondary causes and deciding whether clinical follow-up is warranted.
Electronic screening tools are only as effective as the quality of information recorded in practice databases. We found that family history was poorly documented for the majority of patients and no patients had a record of arcus cornealis or tendinous xanthomata. This highlights the need for increased comprehensiveness of recording in GP databases to enable development of similar electronic tools to increase clinical efficiency. Models such as FAMCAT18 based on indicators routinely available in EMRs may offer an alternative to the DLCNC for use in general practice screening with greater discriminatory power. However, FAMCAT is yet to be externally validated and trialled in a general practice setting.
Electronic screening is also subject to inherent limitations of the electronic system itself. In our study, statin strength/dosage data were not extractable in Best Practice which may explain why the GP review returned higher DLCNC scores (8+) compared with TARB-Ex (ie, the GP was able to apply the cholesterol adjustment based on actual statin dosage whereas TARB-Ex applied the lowest correction for a given medication). Second, where cholesterol corrections were automatically applied, it was deemed that corrections were not required in almost all cases as patients were likely non-compliant with prescribed medication. In addition, in determining potential risk of FH we set a threshold of DLCNC score ≥5. However, it could be argued that those who score 3–4 using DLCNC (also categorised as possible FH) would be missed and we acknowledge this as a limitation.
While electronic screening methods can never be 100% accurate, we have shown that electronic screening can serve as a powerful tool to quickly and accurately identify patients at potential risk of FH for further investigation. Routine electronic screening combined with clinical follow-up with a GP may offer a time-effective and cost-effective systematic approach to identifying new index cases based in primary care. It is by using this retrospective approach combined with a more proactive, opportunistic approach aimed both at identifying new index cases and cascade testing relatives that increased detection of FH may be facilitated in primary care. Proposed methods of shared care11 ,12 will provide an additional29 and sustainable approach to the management of diagnosed FH patients largely within primary care.
Our future research is focused on broader implementation and validation of TARB-Ex across multiple practices and EMR platforms. To date, we have trialled TARB-Ex in eight Australian practices and are in the process of integrating TARB-Ex for MedicalDirector. Evaluation of TARB-Ex's performance in external settings will provide greater insight into the clinical utility of the tool for detecting new index cases of FH.
What is already known on this subject?
Only an estimated 10% of familial hypercholesterolaemia (FH) cases are currently detected.
General practice has been increasingly identified as an optimal setting for FH screening.
Methods of screening need to integrate easily into existing clinical practice.
What might this study add?
Our study provides an easy and accurate method of systematically identifying general practice patients at potential risk of FH for further investigation using routine clinical information recorded in electronic medical records.
How might this impact on clinical practice?
Routine electronic screening combined with clinical follow-up with a general practitioner may offer a time-effective and cost-effective systematic approach to identifying new index cases based in primary care.
We acknowledge the financial support of L Ryan to the General Practice Research Unit.
Contributors LT developed the electronic screening tool, performed data collection, wrote the statistical analysis plan, cleaned and analysed the data and drafted and revised the paper. She is the guarantor. DA-R designed and supervised the study and drafted and revised the paper. WCSP-D drafted and revised the paper. GFW reviewed patient records, performed clinical assessment with patients and revised the draft paper. JP reviewed patient records and revised the draft paper. ML performed data collection, reviewed patient records and revised the draft paper. MB provided statistical input and revised the draft paper. DM monitored data collection and revised the draft paper. MJ performed data collection and revised the draft paper. TB designed and supervised the study, reviewed patient records, performed clinical assessment with patients and revised the draft paper.
Funding This study was supported through the Australian Government's Collaborative Research Networks (CRN) programme to the University of Notre Dame and the University of Notre Dame Australia Research Incentive Scheme (2014R203).
Competing interests None declared.
Ethics approval The University of Notre Dame Australia Human Research Ethics Committee (014017F).
Provenance and peer review Not commissioned; externally peer reviewed.