Table 2

Study designs to study the impact of a prediction model on individuals' and doctors' behaviour or decision-making, and on individuals' health outcomes

Design of impact studyStudy characteristicsExample
(Cluster) randomised trial
  • Comparing outcomes between individuals or care providers randomly assigned to receive/apply management/decisions guided by the prediction model (ie, risk-based management) versus no risk-based-management (care-as-usual)

  • Unbiased comparisons

  • Time consuming and expensive

Quantifying the effects of communication of absolute cardiovascular disease risk and shared decision-making using a simple decision aid for use in family practice consultation35
Stepped-wedge cluster randomised trial
  • Comparing individuals' outcomes between clusters which first apply care-as-usual and subsequently, at randomly allocated time points, risk-based management

  • Unbiased comparisons

  • Useful for complex interventions that can be evaluated during implementation in routine care Time consuming and expensive

Measure the impact of a multifaceted strategy, including a preoperative risk assessment, to prevent the occurrence of postoperative delirium in elderly surgical patients36
Prospective before–after study
  • Comparing individuals' outcomes between those treated conventionally in an earlier period and those treated in a later period after introduction of the prediction model

  • Sensitive to potential time effects and subject differences

  • Time consuming

The PREDICT-CVD programme to investigate whether introduction of integrated electronic decision support based upon the Framingham absolute risk equation improves cardiovascular disease risk assessment37
Decision analytic modelling
  • Combines evidence on the accuracy of model predictions from observational model (external) validation studies, and on the effectiveness of subsequent management from randomised therapeutic trials or meta-analysis

  • Relies on various model inputs and assumptions

  • Less time consuming and low costs

Predicting the impact on a population level on the incidence of CVD-related events over a 5–10-year period, using prediction models (such as the UKPDS and a derivative of the Framingham risk equation)38
Cross-sectional study
  • Comparing care providers' decisions after being randomised to either use or not use the model's predicted risk

  • No subject outcome (no follow-up)

  • Less time consuming and low costs

The AVIATOR study to quantify whether global risk assessment on coronary heart disease leads to different targeted preventive treatments39
Before–after study within the same care providers
  • Care providers are asked to document therapeutic management decisions before and after being ‘exposed’ to a model's predictions

  • No subject outcomes required (no follow-up)

  • Less time consuming and low costs

Effect of using 10-year and lifetime coronary risk information on preventive medication prescriptions as compared with not using these risks40
  • CVD, cardiovascular disease.