Objective To assess differences between cardiovascular disease (CVD) risk estimation methods when applied to a black UK population.
Design Cross-sectional study.
Setting 51 GP practices in Lambeth, south-east London.
Patients 26 370 black and 52 288 white registered patients aged 40–74 years.
Main outcome measures 10-year CVD risk score estimates derived using Framingham, QRISK2, ASSIGN and ETHRISK algorithms. κ measures of agreement between risk scores and age-adjusted black/white mean risk ratios (RR) derived for each score.
Results There was a moderate agreement between the various risk scores for the black population (pooled κ 0.59 (95% CI 0.57 to 0.61) for men and 0.42 (95% CI 0.39 to 0.46) for women). For the white population, agreement was significantly improved (pooled κ 0.74 (95% CI 0.73 to 0.76) for men and 0.51 (95% CI 0.49 to 0.54) for women). Except for QRISK2, each method consistently overpredicted the CVD risk for the black population in comparison with national (Health Survey for England) prevalence figures. QRISK2 estimates were the least divergent from national data, giving a black/white mean RR of 0.73 (95% CI 0.71 to 0.74) for men and 0.85 (95% CI 0.83 to 0.87) for women.
Conclusions The choice of risk estimation method does make a difference to estimates of CVD risk for black patients. The QRISK2 method, which incorporates ethnicity as a risk factor, appears to have the best fit with national data for this population.
Statistics from Altmetric.com
Risk screening now plays a prominent role in the prevention of cardiovascular disease (CVD) in the UK and internationally, with predicted CVD risk the key determinant informing risk management decisions.1 Current guidelines recommend that GPs screen all patients aged over 40 using routinely collected data, prior to a full risk assessment for those considered at high risk.2 Until recently, most risk scores were based on a formula derived from the US Framingham study. Current guidelines no longer recommend this specifically but leave the choice of risk estimation method up to GPs themselves.2 An often highlighted shortcoming of the Framingham method is that this was derived from a predominantly white US middle class sample and may therefore not be applicable to other ethnic, cultural and socioeconomic groups.3 ,4
In the UK, ethnic differences in CVD risk are well documented.4–7 A number of UK-specific risk estimation methods have been devised, some of which reflect ethnic differences. The QRISK2 score, based on routinely collected primary care data, incorporates both patient's ethnicity and a measure of area deprivation along with other risk factors.8 Another score, ETHRISK, also explicitly sets out to account for ethnic differences by recalibrating the original Framingham formula to reflect existing ethnic differences in CVD prevalence and mortality in the UK.9 Another UK-specific score, ASSIGN, based on the Scottish Heart Health Extended Cohort study, also includes a measure of area deprivation but does not explicitly account for ethnic differences.10
Recent national guidelines stress an urgent need for research assessing the relative benefits of different risk estimation methods and, in particular, the relevance of ethnicity as a risk factor.2 Important differences in predicted outcomes have already been found when different estimation methods are applied to the South Asian population.11 However, no similar comparative studies have been carried out for the UK black (African and black Caribbean) population.
We set out to investigate how the above risk estimation methods apply to a UK population with a large proportion of black patients. Our study has two primary aims: (1) to examine the extent to which different estimation methods produce different results for this population; and (2) to assess how well these scores reflect what we already know from national data about ethnic differences in CVD. A secondary aim of the study is to examine CVD risk scores for the Caribbean and African population separately to reflect differences in disease risk for these ethnic subgroups.
Risk factor data from electronic patient records (Lambeth DataNet) for practices in Lambeth were used. This borough in south-east London has the second highest proportion of census-defined ‘black or black British’ residents in the UK at 25.8% (neighbouring Southwark has the highest proportion at 25.9%).12 The database was originally set up to improve both the level and quality of ethnicity coding in GP data and therefore has a high level of ethnicity coding.13 ,14
The study population comprised all (363 331) patients registered at DataNet practices, including all but one of the 52 GP practices in the borough. Data were extracted from local practice computer systems in March 2011. Our analysis was restricted to patients with self-reported ethnicity data and those aged 40–74 years, as this is the age range used in NHS CVD risk assessments.2
Risk scores and risk factor measurements
A range of cardiovascular risk factors are now routinely recorded by GPs in line with the Quality and Outcomes Framework requirements. These cover the preceding 15 months, and we therefore used these and other routine measures collected over the same time period. Patient's self-assigned ethnicity was mapped to UK census ethnic categories.12
We used the published Framingham prediction function to calculate Framingham scores predicting 10-year CVD risk.15 For ASSIGN we used the published algorithm which requires the Scottish index of multiple deprivation (SIMD) as a measure of area deprivation.10 We matched this to an equivalent English deprivation measure using a table of equivalent scores from a recently published study, providing average SIMD scores for each Townsend deprivation quintile.16 For QRISK2 we used the batch processor implementation of the QRISK2 Cardiovascular Disease algorithm V.2.17 For ETHRISK we used the algorithm supplied by the author for the online calculator based on the original paper outlining this method.9 ,18
Further exclusion criteria specific to each risk score were as follows. For each risk score, patients with any prior CVD event including coronary heart disease (CHD), angina and stroke were excluded. For ETHRISK only, patients with a history of diabetes were also excluded. For all risk scores except QRISK2, patients with missing systolic blood pressure, cholesterol data or smoking status were also excluded.
Analysis of data
We carried out a pairwise comparison of the four risk estimation methods to see how they allocated patients to a high CVD risk group. We defined high risk as a probability of 20% or more of a CVD event within the next 10 years—following current national treatment guidelines.2 The κ statistic was used to quantify the degree of agreement over and above the probability of two scores agreeing by chance. For each pair of methods this gives a score between 0 (indicating no agreement) and 1 (indicating perfect agreement).19 We also pooled the results for the different methods using κ adapted for multiple raters, effectively treating each estimation method as a different rater.20
We also derived mean CVD risk ratios using each risk estimation method, comparing the mean risk for each ethnic group with the majority white population. To account for the effect of a skewed CVD risk distribution, we log transformed the scores before calculating the adjusted mean for each group. Taking the anti-log of the adjusted mean differences in log-CVD risk gave us the ratio of geometric mean risk between ethnic groups, which is what we have reported here. Mean risk ratios were calculated separately for men and women and adjusted for age.
Risk estimation methods differ in those they exclude and also the way they handle missing risk factor data. It is possible that these differences could influence the results of the second part of the analysis. We therefore carried this out in two ways. First, we compared mean CVD risk ratios for each ethnic group using all the risk prediction data. Second, we carried out a sensitivity analysis, restricting our comparison to only patients with CVD risk calculated using all four estimation methods.
The results were then compared with age-standardised risk ratios based on national CVD prevalence data taken from the Health Survey for England (HSE). We reanalysed the original HSE data to match the age range of our study population and we report here ethnic differences compared with the white population. Other than this, we have adopted exactly the same methodology used in the original HSE report, including incorporating the original survey weights in our analysis.21 All statistical analyses were performed using Stata (V.11).
We were able to extract records for 124 033 patients in the specified age range, of whom 26 370 (21%) were in the black or black British census category, with 10 973 (9%) in the black Caribbean and 12 215 (10%) in the black African subgroups. The sample included 52 288 (57%) white patients and 13 408 (11%) in other ethnic groups. As no other ethnic group predominated, we concentrated our analysis on black patients.
Table 1 shows some key characteristics of our sample. There were no particularly marked differences between black and white patients other than that a greater proportion of black men and women were living in more deprived areas and were more likely to be diabetic. Also, black women had a higher body mass index and a lower smoking rate compared with white women.
The number of patients for whom a CVD risk could be calculated varied according to the estimation method used. For QRISK2, extensive imputation meant that, in effect, no patient was excluded as a result of missing data. However, for the other three estimation methods, only a minority of cases had complete risk factor data. For ASSIGN, 25 196 (32%) patients, for Framingham 23 112 (29%) patients and for ETHRISK only 19 109 (24%) had all the risk factor data needed to calculate a risk score. ETHRISK had the most missing scores as this measure excludes all patients with diabetes.
We first compared scores to see how they assigned patients to a high or low CVD risk group (see table 2). We found only a moderate agreement between the various CVD risk estimation methods for the black population (pooled κ 0.59 (95% CI 0.57 to 0.61) for men and 0.42 (95% CI 0.39 to 0.46) for women). For the white population, agreement between risk scores was noticeably improved (pooled κ 0.74 (95% CI 0.73 to 0.76) for men and 0.51 (95% CI 0.49 to 0.54) for women). Looking at the ethnic subgroups, overall the scores appeared to diverge most when applied to the black African population (table 2), giving a pooled κ of 0.53 (95% CI 0.49 to 0.56) for men and 0.39 (95% CI 0.35 to 0.44) for women.
Looking at the black population alone, it is apparent that substantially more are classed as high risk under Framingham (1.711, 20.5%) than with QRISK2 (977, 11.8%) (figure 1), whereas Framingham, ASSIGN and ETHRISK show much greater similarity.
When we compared ethnic differences in mean CVD risk for both African and Caribbean groups we found that, apart from QRISK2, all the risk methods consistently overpredicted CVD in comparison with national figures (table 3). For example, Framingham predicts a black/white RR of 1.02 (95% CI 1.00 to 1.04) for men and 1.10 (95% CI 1.07 to 1.12) for women. This contrasts with national data giving an equivalent prevalence ratio of 0.70 (95% CI 0.40 to 1.00) for men and 0.92 (95% CI 0.56 to 1.29) for women. The nearest comparable estimate to national data is QRISK2 which gives a black/white RR of 0.74 (95% CI 0.73 to 0.75) for men and 0.89 (95% CI 0.88 to 0.90) for women. CIs based on the national data are, however, very wide and therefore overlap with all risk score estimates, except in the case of African men. Here the scores appear to overestimate the CVD risk while national data put this group at a significantly reduced risk (0.20 (95% CI 0.00 to 0.50)). In this instance, QRISK2 gives a mean RR of 0.74 (95% CI 0.73 to 0.74), showing a statistically significant difference from other estimates. Although this is still a significant overestimate, it provides the best fit with the national data.
Recent national guidelines have called for research to assess the merits of different risk estimation methods and to account for the added value of including variables such as ethnicity.2 Our study suggests that QRISK2, one of two UK-specific methods to include ethnicity as a factor, provides the best fit for a black UK population based on what is known from national data. The only other score to include ethnicity, ETHRISK, produced very divergent results. It could be argued that the latter score, a recalibration of Framingham, has largely been superseded by QRISK2 which is based on a UK-specific cohort. The same guidelines also call for studies to apply the ASSIGN method to UK populations outside Scotland, as we have done here. The derived ethnic risk ratios using ASSIGN are similar to those derived using the Framingham formula, which may well reflect the absence of ethnicity as a risk factor in both of these scores.
We also found that QRISK2 reallocated to low risk over half the black patients that Framingham would class as high risk. Given that these patients would be most affected by a GP practice changing from Framingham to the increasingly common QRISK2 method, it is perhaps worth looking in more detail at their characteristics. We found that those who were reclassified in this way had a much lower prevalence of diabetes than those who remained a high risk under both scores (31% vs 69%). They were also younger (mean age 60 vs 67 for those remaining high risk), more likely to be male (75% vs 61%), had a higher mean cholesterol ratio (4.0 vs 3.8) and higher mean systolic blood pressure (144 vs 142). All these differences were statistically significant (p<0.05).
Strengths and weaknesses
Our study benefited from a large sample of the ethnic groups in question along with relevant risk factor data which allowed us to make good use of the estimation capabilities of each method. Our data source also has a high level of ethnicity recording in the study area, Lambeth, and reflects patients from a very wide range of socioeconomic backgrounds. The study was, however, restricted to a cross-sectional analysis only, so we were unable to compare predicted risk with patient outcomes. Instead we assessed the performance of the scores by comparing estimated RRs with those already found using national data. We were also restricted to comparing only those estimates for patients for whom relevant risk factor data had been recorded. However, by using electronic patient records for our analysis, our situation is analogous to any practice screening for CVD risk using health records alone. It is, however, possible that the wide variation in the amount of missing data between QRISK2 and the other risk score methods could introduce a bias when we compare scores. To account for this, we also conducted a sensitivity analysis, recalculating ethnic differences for each score and restricting our analysis to only those patients with a complete set of all four risk scores, and found no overall difference in the reported results (see table 4 in online appendix 1). It is also worth noting that our study does not differentiate between types of CVD and therefore cannot reflect important ethnic differences between heart disease and stroke risk. However, current guidelines recommend that GPs concentrate on measuring overall CVD risk2 and newer risk scores such as QRISK2 reflect this, so this is what we have assessed here.
Comparison with previous studies
A number of comparative studies have also looked at the estimation methods covered here.16 ,22 ,23 However, to our knowledge, ours is the first study to compare different CVD risk estimation methods for a black UK population. There have, however, been attempts to assess the performance of Framingham scores alone for this ethnic group. Cappuccio et al6 applied the Framingham formula to patient data in south-west London in order to estimate the overall CVD risk and found that Framingham predicted lower age- and sex-adjusted CVD rates for patients of African origin (10.5 (95% CI 9.7 to 11.2) events per 1000) compared with white patients (11.9 (95% CI 11.0 to 12.7)). This gives an overall ethnic RR of 0.88, which has a better fit with the ethnic prevalence ratio found in national data. This differs from our study which showed that Framingham predicts a significantly higher CVD risk for black patients than for white patients. Another study24 followed a similar approach, applying the Framingham formula for CHD to a range of ethnic groups using cross-sectional data from the HSE, 1998–1999. They also compared age-adjusted mean RRs with national outcome data, as we did, and similarly found that Framingham substantially overestimated the risk for the black Caribbean group.24 For example, they predicted that black Caribbean men have a CHD RR according to Framingham of 0.91 (95% CI 0.83 to 0.99) compared with white men, while the national standardised mortality ratio (SMR) is considerably lower at 0.62 (95% CI 0.58 to 0.67). Similarly, for Caribbean women they report Framingham giving a RR of 0.94 (95% CI 0.74 to 1.19) compared with a national SMR of 0.86 (95% CI 0.77 to 0.96).
We have established that different risk estimation methods produce different results for black Caribbean and African ethnic groups. Our results suggest that the QRISK2 method most closely reflects the ethnic differences seen in national prevalence data. A prospective cohort study is now needed to establish the validity of these different risk estimation methods for black Caribbean and African populations.
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online Appendix 1
Funding The study was supported by a grant from the Guy's and St Thomas' Charity.
Competing interests None.
Ethics approval Ethical approval was given by the South East Research Ethics Committee (07/MREOI/26).
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.