Article Text

PDF

Original article
Individualised prediction of alternate-day aspirin treatment effects on the combined risk of cancer, cardiovascular disease and gastrointestinal bleeding in healthy women
  1. Rob C M van Kruijsdijk1,
  2. Frank L J Visseren1,
  3. Paul M Ridker2,
  4. Johannes A N Dorresteijn1,
  5. Julie E Buring2,
  6. Yolanda van der Graaf3,
  7. Nancy R Cook2
  1. 1Department of Vascular Medicine, University Medical Centre Utrecht, Utrecht, The Netherlands
  2. 2Division of Preventive Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
  3. 3Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
  1. Correspondence to Dr Frank L J Visseren, Department of Vascular Medicine, University Medical Centre Utrecht, Heidelberglaan 100, PO Box 85500, Utrecht 3508 GA, The Netherlands; f.l.j.visseren{at}umcutrecht.nl

Abstract

Background The value of aspirin in primary prevention of cancer and cardiovascular disease (CVD) remains unclear. The aim of this study was to identify women who benefit from alternate-day aspirin with regard to all relevant outcomes, including cancer, CVD and major gastrointestinal bleeding.

Methods Long term follow-up data of 27 939 healthy women with baseline plasma samples in the Women's Health Study, a randomised trial of 100 mg alternate-day aspirin versus placebo, were used to develop competing risks models for individualised prediction of absolute risk reduction of the combination of CVD, cancer and major gastrointestinal bleeding by aspirin.

Results Although aspirin was associated with a modestly decreased 15-year risk of colorectal cancer, CVD, and in some women non-colorectal cancer, aspirin treatment resulted in a negative treatment effect in the majority of women if gastrointestinal bleeding was also taken into account. The excess risk of major gastrointestinal bleeding by aspirin increased with age, but the benefits for colorectal cancer and CVD risk were also greater at higher age. Decision curves indicated that selective treatment of women ≥65 years may improve net benefit compared to treating all, none and prediction-based treatment. The observed 15-year number needed to treat to prevent one event among women ≥65 years was 29 (95% CI 12 to 102).

Conclusions Concurrent evaluation of the absolute effects on cancer, CVD and major gastrointestinal bleeding showed that alternate-day use of low-dose aspirin is ineffective or harmful in the majority of women in primary prevention. Selective treatment of women ≥65 years with aspirin may improve net benefit.

Trial registration number NCT00000479.

  • MALIGNENCY
  • CORONARY ARTERY DISEASE

Statistics from Altmetric.com

Introduction

Emerging data convincingly show that aspirin, in addition to its effects on cardiovascular risk, reduces cancer risk.1–4 Recent meta-analyses of individual patient data from randomised trials of daily aspirin showed a notable decrease in both cancer incidence and mortality, particularly for colorectal cancer.2 ,3 ,5 The protective effects were more pronounced in trials with longer duration of treatment and emerged only after a delay of 5–10 years, depending on the dose used.1–3 ,5 ,6 In contrast to daily aspirin, no effect of alternate-day aspirin on cancer risk was observed in previous analyses of the two largest randomised trials of aspirin, the Women's Health Study (WHS) and the Physicians’ Health Study (PHS).7 ,8 Recently, however, analysis of long term observational follow-up data of the WHS revealed a reduction in colorectal cancer risk in the aspirin group, emerging after a median follow-up of 18 years (HR 0.80, 95% CI 0.67 to 0.97).9

Despite these findings, the role of aspirin in primary prevention remains unclear, as it is uncertain whether the combined benefits for cancer and cardiovascular disease (CVD) outweigh the increase in major bleeding events.4 ,10 The US Food and Drug Administration recently published a consumer update in which the use of aspirin for primary prevention of CVD is discouraged,11 whereas current guidelines, focusing on CVD, recommend to consider use of aspirin prophylaxis for individuals at high cardiovascular risk12 and in those ≥65 years of age, if the benefit for CVD prevention is likely to outweigh the risk of bleeding events.13 ,14 However, for whom the latter is the case, especially if the potential benefits for cancer prevention are also considered, remains to be established.

As treatment effect may be determined by multiple patient characteristics, using models to predict treatment effect for individuals could help to select patients for aspirin treatment.15–20 This would enable clinicians to estimate the response of an individual to aspirin prophylaxis and only treat those who are expected to benefit.

Using data from the WHS, we developed models for predicting aspirin treatment effect (ie, 15-year absolute risk reduction (ARR) of the combination of CVD, cancer and major bleeding events), aimed at identifying initially healthy women who could benefit from aspirin. Moreover, we evaluated which of the following aspirin treatment strategies would lead to the most favourable clinical outcome: treat none, treat everyone, treat only women ≥65 years, and prediction-based treatment.

Methods

The WHS was a randomised trial evaluating the effect of 100 mg alternate-day aspirin compared with placebo for primary prevention of CVD and cancer in 39 876 women ≥45 years of age, without a history of CVD or cancer. Detailed methods and outcomes have been described previously.7 ,9 ,21 ,22 Written informed consent was obtained from all participants and the trial was approved by the Institutional Review Board of Brigham and Women's Hospital. After the end of randomised treatment on 31 March 2004, with an average 10 years of follow-up, participants were invited for further observational follow-up.9 A detailed description of the post-trial follow-up and endpoint ascertainment is provided in online supplementary appendix 1. The present analyses include endpoints accrued and confirmed through 14 March 2012, using data of women who provided an adequate baseline plasma sample (n=27 939).

Model derivation

To obtain individualised predictions of treatment effect of aspirin, proportional subdistribution hazards models23 for four outcomes were developed: (1) CVD (ie, non-fatal myocardial infarction, non-fatal stroke, or death from cardiovascular causes), (2) invasive colorectal cancer; (3) non-colorectal cancer (ie, any invasive neoplasm, excluding colorectal and non-melanoma skin cancer); and (4) major gastrointestinal bleeding. The latter was defined as gastrointestinal bleeding events requiring hospitalisation. Reports of cancer were confirmed by pathology or cytology reports or, rarely, were based on strong clinical and radiologic or laboratory marker evidence.7 ,9 Given that the evidence of a preventive effect of aspirin is most abundant for colorectal cancer, this outcome was modelled apart from other cancers, so that any specific effects of aspirin on colorectal cancer risk could be evaluated separately. To avoid non-additivity of risks for individual endpoints, outcomes were modelled in a competing risks framework, mutually accounting for the events of interest, as well as for death by causes other than CVD, cancer or gastrointestinal bleeding (see online supplementary appendix 2.1).23 ,24 Models were developed for treatment effect prediction at 10 and 15 years. To reduce overfitting, predictors that were deemed to be easily available in clinical practice, including age, smoking status, body mass index, systolic blood pressure, use of blood pressure lowering medication, total cholesterol, high density lipoprotein cholesterol, high sensitivity C-reactive protein, family history of premature coronary heart disease, haemoglobin A1c if diabetic, height, diabetes mellitus, alcohol use, menopausal status, hormone replacement therapy use, family history of cancer and history of dyspepsia, were preselected based on existing literature (see online supplementary appendix 1). The relative treatment effect of aspirin was assumed constant in the main analysis. Findings of effect modification by any risk factors are inconsistent in previous studies3 ,7 ,9 ,21 ,25 although significant effect modification was found by age and smoking for CVD in the WHS.21 To evaluate these potential relative subgroup effects, sensitivity analyses were performed in which treatment interactions were considered (see online supplementary appendix 1).

To obtain individualised ARRs, the models were used to predict the absolute risk of all individual outcomes with and without aspirin. Subsequently, the ARRs were calculated as the difference between the predicted absolute risk with and without aspirin treatment and the ARRs of the individual outcomes were summed to get a total ARR. As some women and/or physicians may consider CVD or cancer diagnosis to be more important than gastrointestinal bleeding, the total ARR was also calculated applying different weights (ie, 0.5, 0.25 and 0.1) for gastrointestinal bleeding.

Model validation

To adjust for overfitting, bootstrap-based uniform shrinkage was applied for the models26 (see online supplementary appendix 1). Discriminatory ability of each model was evaluated using an optimism-corrected estimate of the c-index that is adapted for competing risks.27 Calibration was assessed graphically using calibration plots.

Decision curve analysis20 was used to evaluate whether use of the models for selecting women for aspirin prophylaxis would improve the clinical outcome compared to other treatment strategies, including treating no one, treating all, and treating only women ≥65 years. This method focuses on the effects of (changes in) treatment decisions that result from a treatment strategy and is based on calculation of ‘net benefit’. Calculation of net benefit starts with choosing a treatment threshold, that is the smallest treatment effect (expressed as ARR) at which one would opt for treatment. This treatment threshold can also be expressed as the number-willing-to-treat (NWT), which is the reciprocal of the treatment threshold and can be interpreted as the maximum acceptable number needed to treat (NNT).17 ,19 Subsequently, this threshold is used for weighing the reduction in event rate by a certain treatment strategy against the harms of treatment. As the appropriate NWT is subjective and can vary among different patients and clinicians, net benefit was calculated for 15-year NWT values ranging from infinite to 20 (ie, treatment threshold of 0–5%). The net benefit results were presented graphically as decision curves. Given that no effect of cancers other than colorectal cancer was observed in previous analysis of the WHS, sensitivity analysis were performed in which the treatment effect of aspirin on non-colorectal cancer was assumed null. Further details on the model development and validation are provided in online supplementary appendix 1.

Results

Baseline characteristics of the present study population (n=27 939) are shown in table 1. During the trial (median follow-up of 10.1 years, IQR 9.5–10.8), 604 cases of CVD, 168 colorectal cancer diagnoses, 1832 non-colorectal cancer diagnosis, and 302 gastrointestinal bleedings requiring hospitalisation were recorded. An additional 107 colorectal and 1388 non-colorectal cancer cases were confirmed during the post-trial period (median follow-up:7.2 years, IQR 4.6–7.3).

Table 1

Baseline characteristics of the total study population and according to predicted 15-year ARR of major cardiovascular events, colorectal cancer, non-colorectal cancer, and major gastrointestinal bleeding with aspirin treatment

Model derivation and validation

The computational formulas for 10- and 15-year treatment effect of aspirin are provided in online supplementary appendix 2.2 and 2.3. Discrimination of the 10-year CVD model was good (c-index0.785), whereas the discrimination of the model for colorectal cancer (c-index 0.65), non-colorectal cancer (c-index 0.59), and gastrointestinal bleeding (c-index 0.641) was moderate. The models for 15-year predictions of colorectal and non-colorectal cancer showed similar discriminatory power (c-index 0.655 and 0.582, respectively). Model calibration was generally well balanced (see online supplementary appendix 2.4).

ARR by aspirin

The WHS participants had a median predicted 15-year risk of 11.4% for all adverse outcomes combined (1.5% for CVD, 0.5% for colorectal cancer, 8.7% for non-colorectal cancer, and 0.8% for major gastrointestinal bleeding). The distribution of individualised 15-year ARRs of aspirin are shown in figures 1 and 2 and the ARRs with NNTs with 95% CIs observed in the WHS population and specific subgroups are shown in online supplementary appendix 2.5 and table 2. Overall, there was a small benefit from aspirin treatment with regard to CVD (15-year ARR 0.27%, 95% CI 0.06% to 0.86%, NNT 371) and colorectal cancer (15-year ARR 0.14%, 95% CI 0.02% to 0.59%, NNT 709). No effect on non-colorectal cancer was observed (15-year absolute risk increase (ARI) 0.08%, 95% CI −0.64% to 0.80%, number needed to harm (NNH) 709) and aspirin increased the risk of gastrointestinal bleeding in all women (15-year ARI 0.75%, 95% CI 0.50% to 1.00%, NNH 133). Consequently, aspirin non-significantly increased the median 15-year risk for all outcomes combined by 0.42% (95% CI −0.45% to 1.29%). However, a more beneficial distribution of ARRs was observed if a weight was applied for gastrointestinal bleeding. The 10-year estimates were largely similar, although effects of aspirin were closer to the null (see online supplementary appendix 2.5 and 2.6). A stronger protective effect of aspirin on CVD was observed in women ≥65 years (15-year ARR 3.11%, 95% CI 1.67% to 5.27%, NNT 29). The risk of gastrointestinal bleeding was also increased in this group, but this increase was relatively smaller than the decrease in CVD, especially if bleeding is given less weight than CVD and cancer (see online supplementary appendix 2.7).

Table 2

Observed 15-year ARRs and numbers needed to treat/harm for aspirin

Figure 1

Distribution of predicted 15-year ARR for major cardiovascular events, colorectal cancer, non-colorectal cancer, and major gastrointestinal bleeding with aspirin treatment in the study population. ARR, absolute risk reduction; NNT/NNH, number needed to treat/harm.

Figure 2

Distribution of predicted 15-year ARR for the total of all outcomes (major cardiovascular events, colorectal cancer, non-colorectal cancer, and major gastrointestinal bleeding) applying different weights for gastrointestinal bleeding, in participants in the Women's Health Study. ARR, absolute risk reduction; NNT/NNH, number needed to treat/harm.

The predicted ARR of CVD and, to a lesser degree, of colorectal cancer increased with higher baseline CVD and colorectal cancer risk (see online supplementary appendix 2.8). In contrast, the absolute risk of gastrointestinal bleeding increased notably in women with high baseline risk when on aspirin. Only women with a total baseline risk of >40% for all outcomes would derive benefit from aspirin, although at which baseline risk aspirin yields benefit is dependent on the weight that is applied for bleeding. A similar effect of age on the predicted 15-year ARR was observed, with increasing benefit for CVD and colorectal cancer with higher age. However, the increase in absolute risk of bleeding by aspirin was also stronger in older individuals.

Table 1 displays the characteristics of the study participants by predicted 15-year ARR for the combination of all adverse outcome (<0%, between 0–1%, and ≥1%), calculated with a weight of 0.25 for bleedings. Notably, 66% of women had a negative overall treatment effect. Older age was an important determinant for treatment effect—of the women with a predicted overall treatment effect of ≥1% ARR (NNT 100), 54% were ≥65 years.

Net benefit assessment

Decision curves for evaluating the net benefit of different aspirin treatment strategies with regard to the total outcome, with different weights for gastrointestinal bleeding, are shown in figure 3. Treating all women of ≥65 years is the most favourable treatment strategy if the 15-year NWT is >32 (ie, one is willing to treat 32 women to prevent one event), but the limit is lower if gastrointestinal bleeding is given less weight. If treatment would be reserved for women ≥65 years, the NNT to prevent one adverse event would be 29 (95% CI 12 to 102). Because the models predicted only a small benefit or even harm for the vast majority, and thus almost no women would be selected for treatment, prediction-based treatment yielded similar benefit as treating none over the whole range of treatment thresholds. Decision curves for the individual outcomes (see online supplementary appendix 2.9) show that treating all women ≥65 years results in the highest net benefit for CVD and non-colorectal cancer, although treating none would be the optimal strategy if the NWT is lower than 30 and 50, respectively.

Figure 3

Decision curves for different aspirin treatment strategies, with different weights applied to major gastrointestinal bleeding. (A) No weight (one bleeding is equal to one cardiovascular event or cancer diagnosis). (B) Weight of 0.5 (two bleedings are equal to one cardiovascular event or cancer diagnosis). (C) Weight of 0.25 (four bleedings are equal to one cardiovascular event or cancer diagnosis). (D) Weight of 0.1 (10 bleedings are equal to one cardiovascular event or cancer diagnosis). Reading the net benefit plot starts with choosing a treatment threshold, that is the absolute risk reduction (ARR) at which one would opt for treatment, or number willing to treat (NWT). An NWT of 30 implies that one is willing to treat 30 women to prevent at least 1 event. Since major gastrointestinal bleeding is already incorporated in the total outcome, the treatment threshold is mainly chosen depending on how important one would deem less serious complications, inconvenience of taking pills, and costs. Positive net benefit means that the treatment strategy led to a more favourable trade-off between benefits (observed decrease in event rate) and harms (the proportion of patients receiving treatment weighted by the reciprocal of the treatment threshold). For example, when using a weight of 0.25 for bleeding (C) and an NWT of 30 (treatment of all women with predicted risk reduction of 3.3% or more, ie, a threshold of 3.3%), treating only women ≥65 years of age yields a positive net benefit of observed reduction in event rate—(proportion receiving treatment×treatment threshold)=0.03748−(0.11×0.033)=0.12%—and would be the optimal treatment strategy, whereas prediction-based treatment gives a net benefit of zero (predicted ARR are below the treatment threshold for all women, so equal to treating none) and treating all worsens clinical outcome (negative net benefit).

Sensitivity analyses

Results of sensitivity analyses are provided in online supplementary appendix 3. The predicted ARRs from the models with treatment interactions were more widely distributed, particularly for non-colorectal cancer, with benefit in 48% of the study population and harm caused in the other 52%. When the effect of aspirin on non-colorectal cancer was assumed null, the total ARR tended to be slightly higher. Overall, however, the results from the sensitivity analysis were similar to the main results and, in both scenarios, decision curve analysis indicated that prediction-based treatment was inferior to treating none or treating only women ≥65 years.

Discussion

In the present study, data of the WHS were used to develop models for treatment effect prediction of alternate-day aspirin on the combination of CVD, cancer and major gastrointestinal bleeding in initially healthy women. Although aspirin was associated with a modestly decreased 15-year risk of CVD and colorectal cancer, aspirin treatment resulted in small benefit or even harm in the majority of women if gastrointestinal bleeding were also taken into account. Age was the most important determinant for benefit of aspirin treatment; this was also reflected by the observation that treating only women ≥65 years of age resulted in a higher net benefit with regard to the combined outcomes compared to other treatment strategies, including prediction-based treatment.

Recent findings that both daily and alternate-day aspirin can reduce cancer risk, particularly for colorectal cancer, have reignited the debate on aspirin in primary prevention. Given that aspirin only modestly lowers cardiovascular risk, while increasing the risk of major gastrointestinal bleeding,10 ,25 the benefits for cancer could tip the balance in favour of aspirin in primary prevention. Moreover, it is important to correctly identify those for whom these benefits of aspirin prophylaxis outweigh the harms, and vice versa. Our results indicate that selectively treating women ≥65 years of age may yield the most favourable clinical outcome, given that the harms (ie, minor adverse effects, inconvenience and costs) of treating 32 (or fewer, if one considers CVD or cancer to be more important than major gastrointestinal bleeding) women with aspirin during 15 years are considered to be acceptable to prevent one case of CVD or cancer.This finding is notable, especially since older age was associated with higher bleeding risk on aspirin treatment. However, in many women ≥65 years of age the benefits of aspirin with regard to cancer and particularly CVD risk outweigh the increased bleeding risk, especially if bleeding events are considered to be less important. The finding that the protective effect of aspirin with regard to CVD risk increases with age is in line with results in men from the PHS.28

A previous cost-effectiveness study, evaluating the benefits of daily aspirin with regard to CVD, showed that aspirin could yield net benefit in individuals with a high CVD risk.29 Although we observed that the benefits of aspirin were dependent on CVD risk, selective treatment of women with >10% 10-year CVD risk did not improve overall net benefit and was inferior to selective treatment of women of ≥65 years when the effects on cancer and bleeding were also taken into account.

As the predicted net benefit of aspirin treatment for most women is small, less serious side effects (ie, minor bleeding and peptic ulcers) become important in aspirin treatment decisions. Extrapolating the combined incidence rates of minor gastrointestinal bleeding and peptic ulcers during the trial period results in a 15-year ARR of −3.4%. This means that for every 29 women using alternate-day aspirin during 15 years, one experiences a minor gastrointestinal bleeding or peptic ulcer.

Treatment based on predictions from multivariable models resulted in lower net benefit than treating women ≥65 years of age. This is possibly due to the usage of multiple models, which might increase the probability of misclassification. In particular, the prediction model for non-colorectal cancer showed a slightly unsatisfactory performance. This outcome comprises a heterogeneous group of cancers, which might have led to the introduction of noise by some of the predictors other than age. This unexpected finding emphasises the importance of evaluating different treatment strategies based on their clinical benefit with regard to all relevant outcomes (eg, by means of decision curve analysis). In the sensitivity analysis, no important changes in treatment effect predictions were observed, indicating that the results are robust.

Study limitations

Some study limitations need to be considered. First, the participants of the WHS are generally at low risk due to selection criteria (eg, all female health professionals). This might limit extrapolation of the results to the general population. Secondly, the present analyses only included first events, meaning that, for example, when a participant experienced both CVD and major gastrointestinal bleeding during the study, only the first event was used. In our view, however, this is similar to clinical practice, where, after non-fatal CVD, bleeding or cancer diagnosis the changes in one's medical condition usually call for a new aspirin treatment decision moment. Thirdly, we presented results with differing weights for major gastrointestinal bleeding, because some might consider bleeding events to be less important than CVD or cancer, but, of course, any weight would be arbitrary. However, if the 15-year NWT would be 32 or higher, the weight for bleeding is irrelevant, as for any lower NWT selective treatment of women ≥65 years of age would be the optimal treatment strategy. Lastly, our results may not apply for daily aspirin as the effects on cancer risk occur earlier than those on alternate-day low-dose aspirin use.1 ,3 ,6

Whether aspirin prophylaxis could indeed be beneficial in the elderly is currently being evaluated in a randomised trial (NCT01038583). Meanwhile, simultaneous evaluation of absolute treatment effects on all relevant outcomes on an individual patient level such as presented in this study, rather than evaluating each outcome at a time on a group level, could provide a sensible approach to determine the value of aspirin in primary prevention.

Conclusions

Alternate-day use of low-dose aspirin for primary prevention is ineffective or harmful in the majority of women with regard to the combined risk of CVD, cancer and major gastrointestinal bleeding. Age is the most important determinant of aspirin treatment effect, and the protective effects of aspirin with regard to CVD increased with age. Although the excess risk of major gastrointestinal bleeding by aspirin is higher in women ≥65 years of age, selective treatment of this group is may improve net benefit.

Key messages

What is known on this subject?

  • Recent evidence suggests that long term use of alternate-day low-dose aspirin may reduce risk for colorectal cancer in healthy women. The value of aspirin in primary prevention, however, remains uncertain, as it is unclear for whom the benefits for the combination of cancer and cardiovascular disease (CVD) outweigh the increase in major bleeding risk.

What might this study add?

  • This study shows that although aspirin is associated with a modestly decreased 15-year risk of CVD and colorectal cancer, aspirin treatment results in small overall benefit or even harm in the majority of women if gastrointestinal bleeding is also taken into account. Age is the most important determinant for benefit of aspirin treatment. Treating only women ≥65 years of age yielded the highest net benefit with regard to the combined outcomes when compared to treating all women, treating none, and prediction-based treatment.

How might this impact on clinical practice?

  • For the majority of women in primary prevention, the long term benefits of alternate-day low-dose aspirin with regard to cancer and CVD do not outweigh the increase in major gastrointestinal bleeding. Selective treatment of women ≥65 years of age with aspirin may improve net benefit.

Acknowledgments

We gratefully acknowledge the support from Eunjung Kim and M Vinayaga Moorthy in assembling and validating the data. Furthermore, we thank the staff of SURFsara (https://www.surfsara.nl/) and HPC-systems (University Medical Centre Utrecht) for providing infrastructure for high performance computing and support in optimising our analyses.

References

View Abstract

Footnotes

  • Contributors RCMvK conceived the research question, designed and carried out the data analyses, interpreted the results and drafted the manuscript. FLJV conceived the research question, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content. PMR conceived the research question, collected the data, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content. JAND conceived the research question, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content. JEB is the principal investigator of the Women's’ Health Study, collected the data and revised the manuscript for important intellectual content. YvdG conceived the research question, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content. NRC conceived the research question, collected the data, designed the data analyses, interpreted the results and revised the manuscript for important intellectual content.

  • Funding The Women's Health Study was supported by grants HL043851, HL080467, HL099355, and CA047988 from the National Heart, Lung, and Blood Institute and the National Cancer Institute, Bethesda, Maryland. This study was financially supported by ZonMw, the Netherlands Organization for Health Research and Development, grant no. 836011027.

  • Competing interests PMR receives investigator initiated research support from the National Institutes of Health, Amgen, Novartis, Pfizer, and AstraZeneca; has served as a consultant to ISIS, Vascular Biogenics, Boston Heart, Jannson, and Genzyme; and is listed as a co-inventor on patents held by the Brigham and Women's Hospital that relate to the use of inflammatory biomarkers in cardiovascular disease and diabetes that have been licensed to Siemens and AstraZeneca. FLJV's department receives grant support from the Netherlands Organisation for Health Research and Development, and the Catharijne Foundation Utrecht; and speaker fees from Merck.

  • Ethics approval The Institutional Review Board of Brigham and Women's Hospital.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Data set: not available. Study protocol and statistical code: available from the authors upon written request. The authors have written an R-function for decision curve analyses for models that apply the results of a clinical trial20 which is also available on request (email: R.C.M.vanKruijsdijk@umcutrecht.nl).

  • Patient Consent Obtained.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Linked Articles