Results

We received 806 risk requests during the study. Nine patients were excluded because they were aged < 30 years, 56 because they were aged > 70 years, five because they were known to have left ventricular hypertrophy, and 45 because the request forms were not completed in full, which precluded risk calculation. Data on the remaining 691 patients were evaluated (table 2). Of these, 229 (33.1%) had a serum cholesterol concentration of ⩾ 6.5 mmol/l, 160 (23.2%) were hypertensive (systolic blood pressure ⩾ 160 mm Hg), 138 (20.0%) had diabetes mellitus, and 145 (21.0%) were current cigarette smokers. Overall, 421 patients (60.9%) had one or more cardiovascular risk factors. The median 10 year projected coronary heart disease risk was 12.6% (range 0.1–48.3%) and median five year projected cardiovascular disease risk was 8.0% (range 0.1–46.7%). In 291 individuals (42%) the projected 10 year coronary disease risk was ⩾ 15%, and in 59 (8.5%) it was ⩾ 30%. In 280 individuals (40.5%) the projected five year cardiovascular disease risk was ⩾ 10%, and in 85 (12.3%) it was ⩾ 20%. There was a strong correlation between projected coronary disease and cardiovascular disease risks (R ² = 0.84), as the Framingham equations for these use the same risk factors but with different weightings. The regression equation between the projected five year cardiovascular disease risk (CVD5) and 10 year coronary heart disease risk (CHD10) was CVD5 = CVD10*0.661 + 0.459, and hence 10 year coronary disease risks of 30% and 15% are equivalent to five year cardiovascular disease risks of 20.3% and 10.7%, respectively. Sensitivity, specificity, positive and negative predictive values, and their respective 95% CIs for the relevant risk thresholds are shown for all of the tables and charts grouped according to their design (tables 3, 4, 5, and6).

The original Sheffield tables have a low sensitivity and positive predictive value, such that only 40% of those eligible for cholesterol lowering treatment would be identified, and of those so classified, one in four actually have calculated risks of < 30% (table 3). They do, however, have a high specificity so that the false positive rate is low. The patient exclusions (serum cholesterol < 5.5 mmol/l and systolic blood pressure > 160 mm Hg) suggested by the authors of the original Sheffield tables reduce the numbers in whom risk can be assessed (370 of 691, 53.5%) but do not significantly improve sensitivity, specificity, or positive and negative predictive values. Following these patient exclusions, fewer true positives are identified (17 v 28). The changes introduced in the modified Sheffield tables significantly improve sensitivity at the projected annual risk threshold of ⩾ 3% (10 year ⩾ 30%) but reduce specificity. The positive predictive value is similar to that of the original tables, such that between one in three and one in four of those classified as having annual coronary heart disease risks of ⩾ 3% actually have lower calculated risks. The modified Sheffield tables also permit assessment at an annual risk threshold of ⩾ 1.5% (10 year ⩾ 15%). Specificity is significantly worse than at the ⩾ 3% risk level, and indeed is among the lowest of all the tabular and chart systems. Sensitivity is the same at both thresholds.

Despite including the patient's HDL cholesterol and blood pressure, the New Zealand tables have similar sensitivities and specificities (table 4) to the original Sheffield tables at the equivalent 30% 10 year coronary disease and 20% five year cardiovascular disease risk levels. At the 10% five year cardiovascular disease risk threshold, specificity is significantly lower, and although sensitivity is higher this is not statistically significant. The updated New Zealand tables have significantly better sensitivities at both risk thresholds than the original tables, although this is achieved at the expense of worse specificities. The European tables, with a similar design to those from New Zealand, have better sensitivity than the original Sheffield and New Zealand tables but their specificity is low and significantly worse than most of the other risk assessment methods (table 4). Their positive predictive value is extremely low, such that between one in two to one in three of those identified as having a coronary disease risk of ⩾ 20% actually have lower calculated risks.

The charts produced jointly by the British Cardiac, Hypertension, and Hyperlipidaemia Societies appear to have significantly better specificities at both the ⩾ 15% and ⩾ 30% 10 year coronary heart disease risk levels (table 5) than the modified Sheffield tables. The higher specificity of these charts is achieved at the expense of worse sensitivity, which is significantly lower than that of the modified Sheffield tables at the ⩾ 15% 10 year risk level, but not quite significantly so at the ⩾ 30% 10 year risk level. These charts have the same specificities as the New Zealand tables, the same sensitivity at the equivalent 30% 10 year coronary heart disease and 20% five year cardiovascular disease risk levels, but a higher sensitivity at the 15% 10 year coronary disease/10% five year cardiovascular disease risk level. Recently published modifications to the charts have no significant effects on sensitivity or specificity at either risk threshold (table 5).

Of the categorical methods of risk assessment, that published by the Framingham investigators has the best performance (table 6), with specificities equivalent to those of the modified Sheffield tables and sensitivities equivalent to the joint British charts. The Canadian method does not identify those with risks ⩾ 30% well, but performs better at the ⩾ 15% risk threshold, with sensitivities and specificities similar to those of the modified Sheffield tables (table6).

Discussion

Quantitative assessment of projected coronary heart disease/cardiovascular disease risk appears now to be the accepted basis for the use of drugs in the primary prevention of cardiovascular disease.15 This concept was initially established for cholesterol lowering drugs3 ,16 and has been extended to the use of antihypertensive agents and aspirin.1 ,17 ,18The Framingham equations are the most widely accepted method for projecting cardiovascular disease/coronary disease risks, and are used in the British, European, and New Zealand guidelines. The equations have been validated retrospectively in British subjects.19 ,20 Quantitative risks calculated from such equations are often described as “absolute”, but it has been argued that this term is a misnomer as it implies a sense of certainty that cannot be applied to prospective risk prediction.21 The Framingham equations have actual or potential defects as they do not include several well established risk factors, may not apply to ethnic groups other than north Europeans, and have not been validated in a prospective fashion. We have therefore used the term “projected” rather than “absolute” risk to indicate the uncertainty of the risk prediction methodology.

Several tabular and graphical formats of the Framingham equations have been developed to avoid the need for computation of risks in clinical practice. The authors of the risk tables or charts have generally assumed that risk estimates obtained from them will agree with risks calculated using the Framingham equations. The construction of tabular and graphical methods has required simplifications of the Framingham equations which do not necessarily ensure that risk estimates will be the same as risk calculations. As risk projection is an uncertain process it would be unfortunate were the tables or charts to introduce additional errors. It is surprising that with the exception of the original and modified Sheffield tables5 ,13 the performance of the risk assessment methods has not been evaluated. The predictive value of a screening test is usually dependent upon the prevalence of the condition in the population. We believe that a cohort of primary care patients being assessed for coronary heart disease prevention is the most appropriate group in which to evaluate the tables or charts, rather than in higher risk patients attending hypertension,13 diabetes,14 or lipid clinics.22 Our group of patients is likely to be typical of those who are screened in primary care for prevention of coronary heart disease and cardiovascular disease.

It is clearly desirable that risk tables and charts should have a high sensitivity (true positive rate) to identify the maximum number of those who should be considered for drug treatment. It is also important that they have as low a false positive rate (defined as 100 − specificity) as possible, so that they do not incorrectly identify subjects for treatment. The false positive rate and the specificity are of central importance when risk assessment methods are applied to a population in which the prevalence of treatable risk is relatively low, as in the general population,5 or even in those who are believed to be at high risk by their primary care physicians. UK guidelines for the use of statins have been predicated in part by a consideration of the affordability of drug treatment,23 and hence treatment thresholds have been set at a relatively high level, comparable to risks observed in those with existing coronary heart disease. Risk assessment methods that have low specificities and high false positive rates necessarily increase total prescribing costs.

The original Sheffield tables are arguably the best known to UK practitioners, having been circulated with the Drug and Therapeutics Bulletin 24 and by the chief medical officer.16 We have previously argued that these tables would be inaccurate as they ignore the individual's HDL cholesterol concentration.25 Subsequent evaluations of the tables in patients with hypertension,13hyperlipidaemia,22 diabetes mellitus,14 and now the data reported here, all confirm that the original tables identify only a minority of those whose projected coronary heart disease risks qualify them for statin treatment. Moreover, the patient exclusions recommended by the Sheffield tables appear to be unnecessary. Modifying the Sheffield tables so that they use cholesterol to HDL ratios rather than total cholesterol, and a different definition of hypertension, greatly improves their sensitivity but significantly worsens specificity. The low specificity leads to a false positive rate that is approximately equal to the true positive rate. Our data are similar, although not identical, to those from an evaluation of the modified Sheffield tables in the general population.5 That study showed sensitivities that were lower, though not significantly so, than those we have found, but specificities that were significantly higher. The reason for this is not clear as the only apparent difference between the two study cohorts was in their average projected level of risk, and this should not affect sensitivities or specificities.

The New Zealand tables include both cholesterol to HDL ratios and measurements of blood pressure, and thereby have been claimed to be inherently superior to the original Sheffield tables, although no evidence was produced to substantiate this claim.7 In contrast, we have found that the sensitivities and specificities of the New Zealand and original Sheffield tables at equivalent projected cardiovascular disease/coronary disease risk thresholds are the same. The relatively poor performance of the New Zealand tables may be related to the use of cholesterol to HDL ratios as whole numbers and blood pressures in 20 mm Hg bands, which necessarily requires an individual's values to be rounded to the nearest level in the table and may result in misclassification. Although revisions to the New Zealand tables improve sensitivities, specificities are considerably worse, such that false positives will far outnumber true positives. The revised tables cannot be recommended. The European tables with a similar format also have a very low specificity, which appears to reflect the use of assumed low HDL concentrations which will tend to overestimate risk levels systematically. The Canadian graphical risk assessment method also systematically underestimates coronary disease risk at the ⩾ 30% projected 10 year risk threshold, and although performance is better at the ⩾ 15% threshold, there is no reason to advocate the use of this system. The Framingham categorical method has sensitivities equivalent to those of the joint British recommendations charts but lower specificities; it would not therefore be the method of choice for primary care patients.

The charts produced jointly by the British Cardiac, Hypertension, and Hyperlipidaemia Societies have particularly high specificities at both the ⩾ 30% and ⩾ 15% projected 10 year risk thresholds, such that the false positive rate is only 3/1000 at the higher threshold, and 3/100 at the lower threshold. These specificities are among the highest of the risk assessment methods and their sensitivities are better than the other tables (original Sheffield and New Zealand) with equivalent specificities. Revisions to the charts appear to increase sensitivity without significant loss of specificity.

An audit of clinicians' opinions on the use of tabular or chart risk assessment methods suggested that the joint British recommendations charts and the New Zealand tables were preferred to the Sheffield tables.26 The modified joint British recommendations charts appear to represent the best combination of specificity, sensitivity, and ease of use, and therefore should be the method of choice for those who elect to use tables and charts. It has been suggested, however, that the direct calculation of coronary heart disease and cardiovascular disease risks with the Framingham equations programmed for personal27 ,28 or laboratory computers14 is practicable, and also desirable because the accuracy of the risk assessment with respect to the Framingham equation is assured. Such approaches would target drugs most efficiently.