Commentary: Statistical problems

Underestimation of risk of ischaemic heart disease

Following an old but increasingly popular tradition,6 Law and colleagues corrected the association between cholesterol concentration and risk of ischaemic heart disease for the attenuation (or regression dilution) due to measurement imprecision. Since cholesterol concentrations vary over time, prospective studies relating future risk of ischaemic heart disease to a single measurement will underestimate the true strength of the association. The adjustment is used because few epidemiological studies have obtained repeat measurements. As Spearman noted in 1904,6 however, such adjustments should be used carefully because they are not free of assumptions. They could be misleading if recent levels of a risk factor are more important determinants of disease than earlier levels; if peaks in the level of the risk factor, rather than the usual value, are the important aetiological factor; or if changes in risk factor levels are important.7 Some studies have been able to examine the association of average cholesterol concentration with risk of ischaemic heart disease directly.^8,9 These have indicated that although the correction factors may overestimate the strength of the association between usual cholesterol concentration and risk of ischaemic heart disease, repeated cholesterol measurements identify people at increased risk of disease better than a single measurement.

Law et al have introduced an extension of the correction for attenuation, which they called correction for surrogate measurement bias. Since it is the low density lipoprotein component of total cholesterol which is considered to increase the risk of ischaemic heart disease they argue that the association of total cholesterol with ischaemic heart disease underestimates the underlying association of low density lipoprotein cholesterol with ischaemic heart disease. They determined the degree to which total cholesterol serves as a proxy measure of low density lipoprotein and used this to inflate the relation between total cholesterol concentration and ischaemic heart disease. This is perhaps an unhappy addition to the statistical armoury, particularly since there are now many studies which have directly examined the association between low density lipoprotein cholesterol and risk of ischaemic heart disease.*RF 10-13* The results of these direct comparisons undermine the argument for applying the correction for surrogate measurement bias.^12,13

Furthermore, it is unclear why we should stop the correction for surrogate measurement bias at low density lipoprotein. The atherogenic component of low density lipoprotein cholesterol is thought to be oxidised low density lipoprotein,14 and measurements of low density lipoprotein will serve only as proxies for this. Thus a further magnification of the risk associations should be introduced to correct this surrogate measurement bias. Once this adjustment has been made we must consider the fact that the presence of oxidised low density lipoprotein is only a surrogate marker of the degree to which unregulated scavenger receptors take up oxidised low density lipoprotein.14 After a few more corrections for surrogate measurement bias the association becomes implausibly large, making the problem of this procedure clear. There is, unfortunately, no statistical substitute for actually measuring what you want to study, especially when trying to quantify risk precisely.

Safety of cholesterol lowering drugs

Since lowering cholesterol concentration reduces the incidence of ischaemic heart disease and little evidence exists that naturally low cholesterol concentrations are themselves harmful^15,16 the safety aspects of interventions used to lower cholesterol become paramount. It is important in this case to separate the effects of cholesterol reduction from the effects of the treatments used to lower cholesterol.17

We have updated a previous meta-analysis to attempt to make sense of the different interpretations that have been made of the clinical trial data.17 As Law et al show, increased mortality from causes other than ischaemic heart disease is seen in the trials of cholesterol lowering drugs. Since this increase is not associated with the degree of cholesterol lowering achieved and is confined to drug trials, it seems that it is the means of bringing about lower cholesterol concentration that is important rather than the low cholesterol concentrations themselves. Furthermore, the adverse effect is restricted to trials with long follow up, which would be expected if it is related to long term ingestion of the drug. For the trials of non-hormonal drugs with short follow up (less than four years) the odds ratio for mortality from causes other than ischaemic heart disease is 0.79 (95% confidence interval 0.55 to 1.15; P=0.2). For trials with longer follow up the odds ratio is 1.23 (1.07 to 1.45; P=0.005).

Law et al dismiss the increased mortality from causes other than ischaemic heart disease in the drug trials by suggesting that there was a chance increase in three primary prevention trials: the World Health Organisation clofibrate, lipid research clinics, and Helsinki studies.*RF 18-20* The unequivocal statement that the increase is due to chance may seem odd to readers brought up to believe that it is not possible to prove a null hypothesis and certainly not a type I error. These three trials contribute around two thirds of the person years of follow up and 60% of the deaths from causes other than ischaemic heart disease in the drug trials Law et al include. It is unjustified to dismiss these results as chance based on abandoning intention to treat analyses and asymmetrical case review. They did not attempt to discover what could have been fortuitous findings in trials with favourable results, and it is obvious that reviewing and rejecting results in one direction but not the other represents a bias.

In the previous analysis,17 benefit - in terms of a favourable effect on all cause mortality - was restricted to patients at particularly high risk of death from ischaemic heart disease. This is not simply due to increased mortality in trials which recruited low risk subjects, as Law et al imply, but is a graded association of increasing benefit with increasing risk of ischaemic heart disease. Such an association would be seen if a small adverse effect is balanced by a proportionate reduction in incidence of ischaemic heart disease. This would translate into greater absolute benefit being seen for patients at higher initial risk of ischaemic heart disease.

Because of their size, the three trials that Law et al consider to have produced anomalous findings contribute greatly to the analysis. Despite this the slope between mortality outcome in trials (log odds ratio for mortality between treatment and control group) and the risk of ischaemic heart disease for subjects in the trials (indexed by the mortality from ischaemic heart disease per 1000 person years in the control group of the trials) is little altered by excluding these trials, although, not surprisingly, the significance is. The slope is -0.0074 (P=0.0009) with all non-hormonal drug trials included and -0.0070 (P=0.08) when the three large trials are excluded.

When these three trials are considered as a group their results show a significantly (P=0.005) less favourable effect on all cause mortality than the other non-hormonal drug trials. This can be explained by the fact that they all recruited low risk participants. The apparent unfavourable outcome disappears (P=0.9) once the level of risk of the participants is taken into account. These could simply be trials which recruited subjects at low risk of ischaemic heart disease, in whom the benefit to risk ratio for treatment was low and adverse effects could therefore become evident.

Implications of targeted intervention

A strategy aimed at identifying people with raised cholesterol concentrations would lead to large numbers of people receiving dietary counselling aimed at reducing cholesterol concentrations. Since individual dietary counselling often achieves disappointing reductions in cholesterol many of these attempts would be unsuccessful.^21,22 The prescription of cholesterol lowering drugs would probably follow. Large increases in prescriptions for such drugs in the United Kingdom over the past five years suggest that such a scenario could be imminent.17 In the United States, where cholesterol lowering with drugs is given more priority, around 6% of the population aged 50 and over were prescribed these drugs in 1988-90.23

Lowering cholesterol concentration produces greater benefits for those at higher risk of ischaemic heart disease, an effect which is even more pronounced when absolute risk rather than relative risk is examined.24 The table shows the number of patients who would need to be treated with drugs for a year to prevent one death, estimated from the data from trials that have been done or from a best case assumption of a 20% reduction in risk of death from ischaemic heart disease and no counterbalancing effect on death from other causes.

The real data show an adverse effect for patients who are not at high risk of death from ischaemic heart disease. However, even in the best case large numbers of patients with a risk of death from ischaemic heart disease of 1-5/1000 person years would need to be treated to prevent one death. This range of risk includes many groups of patients whom current guidelines identify as candidates for drug treatment. Clearly the costs - in terms of patient time, the psychological effects of taking treatment to prevent a serious disease, the side effects of drug treatment, and direct expenses for medical care and the drugs - will be great compared with the potential benefits.

Conclusion

Does the current evidence regarding cholesterol, cholesterol lowering, and mortality suggest that we should change direction1 or carry on with current practice?2 There is no good evidence that naturally having a low blood cholesterol concentration is harmful. Nor does reducing cholesterol concentration without using drugs seem to be harmful. Reduction of cholesterol concentration will have a small benefit for most individuals, but in whole populations many deaths could be delayed. People and communities should be given this information and the means to adapt to it as they see fit.

On the other hand, strategies to identify individuals with high cholesterol concentrations will be associated with the usual material and psychological costs of screening. Many will fail to respond adequately to dietary intervention and become candidates for lifelong drug treatment. This will be accompanied by high medical costs and the usual array of minor side effects. Except for patients at greatly increased risk of ischaemic heart disease, current evidence suggests such drug treatment will be associated with, at best, little benefit, if not an adverse overall effect on mortality. Here we need a change of direction - to turn away from the identification and drug treatment of asymptomatic people with isolated mildly or moderately raised cholesterol concentrations.

ME is supported by the Swiss National Science Foundation.